-
- Veeam Legend
- Posts: 945
- Liked: 221 times
- Joined: Jul 19, 2016 8:39 am
- Full Name: Michael
- Location: Rheintal, Austria
- Contact:
object storage integrity check
Hi everybody,
cloud object storage is much more resilent than self-managed on prem storage, I've already read an article about that. But nevertheless, it would be good to be able to run some kind of integrity checks aka SureBackup / block-validation-tool to make sure that your data on object storage is valid and usable in case if you need it.
I've already had the case that a bug in veeam-logic has silently wiped out needed/existing blocks from object storage and I only recognized it due to a FLR - otherwise I would not have noticed that. Of course, there is this new immutability feature but maybe you wouldn't "seal" all blocks for months/years and even then if there is a corruption on storage side, you wouldn't notice it.
So, are there any plans for the short- or medium-term to be able to do some kind of block-validation or even SureBackup on an object storage?
Thanks for letting me know!
cloud object storage is much more resilent than self-managed on prem storage, I've already read an article about that. But nevertheless, it would be good to be able to run some kind of integrity checks aka SureBackup / block-validation-tool to make sure that your data on object storage is valid and usable in case if you need it.
I've already had the case that a bug in veeam-logic has silently wiped out needed/existing blocks from object storage and I only recognized it due to a FLR - otherwise I would not have noticed that. Of course, there is this new immutability feature but maybe you wouldn't "seal" all blocks for months/years and even then if there is a corruption on storage side, you wouldn't notice it.
So, are there any plans for the short- or medium-term to be able to do some kind of block-validation or even SureBackup on an object storage?
Thanks for letting me know!
-
- Veeam Software
- Posts: 492
- Liked: 175 times
- Joined: Jul 21, 2015 12:38 pm
- Full Name: Dustin Albertson
- Contact:
Re: object storage integrity check
Possibly, but one side effect of this would be API calls. In on prem instances of object storage this may not be an issue, but with cloud based object adding daily, weekly, monthly health checks of each object will have a big effect on costing.
Dustin Albertson | Director of Product Management - Cloud & Applications | Veeam Product Management, Alliances
-
- Chief Product Officer
- Posts: 31806
- Liked: 7300 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
-
- Veeam Legend
- Posts: 945
- Liked: 221 times
- Joined: Jul 19, 2016 8:39 am
- Full Name: Michael
- Location: Rheintal, Austria
- Contact:
Re: object storage integrity check
Of course. But if your provider wouldn't charge for that (for instance wasabi), then it wouldn't matter
-
- Veeam Legend
- Posts: 945
- Liked: 221 times
- Joined: Jul 19, 2016 8:39 am
- Full Name: Michael
- Location: Rheintal, Austria
- Contact:
-
- Chief Product Officer
- Posts: 31806
- Liked: 7300 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: object storage integrity check
@Dima P. can you please check with SureBackup developers on this limitation? Thanks.
-
- Veeam Legend
- Posts: 945
- Liked: 221 times
- Joined: Jul 19, 2016 8:39 am
- Full Name: Michael
- Location: Rheintal, Austria
- Contact:
Re: object storage integrity check
Strange, just wannted to double-check and it looks like that I can choose some of my offloaded backups but not all of them?? Looks like the "normal" backups are available but not the those which were created by a BCJ...
-
- Product Manager
- Posts: 14720
- Liked: 1705 times
- Joined: Feb 04, 2013 2:07 pm
- Full Name: Dmitry Popov
- Location: Prague
- Contact:
Re: object storage integrity check
Michael,
Checking with RnD folks, stay tuned!
Checking with RnD folks, stay tuned!
-
- Product Manager
- Posts: 14720
- Liked: 1705 times
- Joined: Feb 04, 2013 2:07 pm
- Full Name: Dmitry Popov
- Location: Prague
- Contact:
Re: object storage integrity check
Michael,
We've checked in the lab and found no issues with the simple setup. Can you please raise a support case and share the ID with us, then I'll ask QA team to review the case details. Cheers!
We've checked in the lab and found no issues with the simple setup. Can you please raise a support case and share the ID with us, then I'll ask QA team to review the case details. Cheers!
-
- Veeam Legend
- Posts: 945
- Liked: 221 times
- Joined: Jul 19, 2016 8:39 am
- Full Name: Michael
- Location: Rheintal, Austria
- Contact:
Re: object storage integrity check
Thanks Dmitry for the update. Just to make sure that it is a bug on our side:
Can you please confirm that in your lab you are able to add a vm to an application group from a copy job? I can only choose from "normal" backup jobs.
Can you please confirm that in your lab you are able to add a vm to an application group from a copy job? I can only choose from "normal" backup jobs.
-
- Product Manager
- Posts: 14720
- Liked: 1705 times
- Joined: Feb 04, 2013 2:07 pm
- Full Name: Dmitry Popov
- Location: Prague
- Contact:
Re: object storage integrity check
Michael,
Update: looks like a limitation of application groups - you cannot use vm from backup file created by backup copy job in application groups. Cheers!
Update: looks like a limitation of application groups - you cannot use vm from backup file created by backup copy job in application groups. Cheers!
-
- Veeam Legend
- Posts: 945
- Liked: 221 times
- Joined: Jul 19, 2016 8:39 am
- Full Name: Michael
- Location: Rheintal, Austria
- Contact:
Re: object storage integrity check
Oh deer, could have checked it on my own... Anyway, now we're at the beginning: How can I now validate these contents on an object storage?
-
- Chief Product Officer
- Posts: 31806
- Liked: 7300 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: object storage integrity check
Well, Dima already confirmed above there are no issues with using SureBackup against backups residing on object storage. So, at the very least you should be able to perform validation by importing those backups to another Veeam Backup & Replication install.
-
- Veeam Legend
- Posts: 945
- Liked: 221 times
- Joined: Jul 19, 2016 8:39 am
- Full Name: Michael
- Location: Rheintal, Austria
- Contact:
Re: object storage integrity check
Yeah but does that really change the situation? I mean if I import those backups within another veeam instance, they are still offloaded BCJ's. Can I then use them for SureBackup? BTW: Will I need a second veeam license then because SureBackup isn't available in the community edition? What is the reason that BCJ's cannot be used in combination with SureBackup/Application groups?
And what happens when I start the backup validator from the commandline, will it then check the offloaded blocks or only the metadata?
Thanks!
And what happens when I start the backup validator from the commandline, will it then check the offloaded blocks or only the metadata?
Thanks!
-
- Chief Product Officer
- Posts: 31806
- Liked: 7300 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: object storage integrity check
Yes, you should be able to use them for SureBackup. I believe the current limitation around BCJ is pretty "artificial" anyway, to help avoid a bunch of bugs with Backup Copy and SureBackup jobs overlapping with one another.
Yes, you will need to install a license in the second Veeam backup server as well.
Good question on Backup Validator. This needs to be tested, but there's a good chance that it will be checking the offloaded blocks too, because blocks location is transparent for the data mover.
Yes, you will need to install a license in the second Veeam backup server as well.
Good question on Backup Validator. This needs to be tested, but there's a good chance that it will be checking the offloaded blocks too, because blocks location is transparent for the data mover.
-
- Veeam Legend
- Posts: 945
- Liked: 221 times
- Joined: Jul 19, 2016 8:39 am
- Full Name: Michael
- Location: Rheintal, Austria
- Contact:
Re: object storage integrity check
Can I use the same license or will I have to purchase one?Yes, you will need to install a license in the second Veeam backup server as well.
-
- Chief Product Officer
- Posts: 31806
- Liked: 7300 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: object storage integrity check
Of course, you can use the same license. Our licensing policy allows for multiple installs with the same license file as long as the total instance/socket usage is not exceeded across all installs. Besides, you cannot purchase a zero instance/socket license anyway which would be required here considering this second installs does not need to back anything up.
-
- Veeam Legend
- Posts: 945
- Liked: 221 times
- Joined: Jul 19, 2016 8:39 am
- Full Name: Michael
- Location: Rheintal, Austria
- Contact:
Re: object storage integrity check
Ok, that's great, thanks Anton. Now since there's an answer for my question, I'd like to ask the famous question: Will SureBackup or block validation via GUI / schedule / automation from a BCJ on an object storage ever be considered and therefore be found in future releases?
-
- Chief Product Officer
- Posts: 31806
- Liked: 7300 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: object storage integrity check
As always, it will depend on the number of requests, as we have to prioritize requests that are in high demand over those that are only requested by a few customers. As it stands right now, this is the only such request in almost 1.5 years since we shipped object storage support. And frankly speaking, I don't expect too many customers doing this with object storage just because of egress and transaction costs.
-
- Veeam Legend
- Posts: 945
- Liked: 221 times
- Joined: Jul 19, 2016 8:39 am
- Full Name: Michael
- Location: Rheintal, Austria
- Contact:
Re: object storage integrity check
Yeah, I understand. I think there's a good explanation why I'm the first in 1.5 years who has this request: Unfortunately I had multiple issues in combination with object storage just because the feature itself is/was quite new. I also lost backups because veeam did somehow purge them due to another issue. Within the console, everything looked normal but when I did a restore it failed and showed me something like "block missing".
Now when you never do validation checks, you would never realize it until you need it... I'm sure that all these issues will get sorted out and that we will have a very stable integration just as we are used to the other integrations. But issues can always occur and as everyone knows: Facing them during restore is the worst moment ever.
Now when you never do validation checks, you would never realize it until you need it... I'm sure that all these issues will get sorted out and that we will have a very stable integration just as we are used to the other integrations. But issues can always occur and as everyone knows: Facing them during restore is the worst moment ever.
-
- Veeam Legend
- Posts: 945
- Liked: 221 times
- Joined: Jul 19, 2016 8:39 am
- Full Name: Michael
- Location: Rheintal, Austria
- Contact:
Re: object storage integrity check
Just wannted to update this thread. We're copying all data to (cloud) object storage with immutability - I love it. Every now and then I import the bucket on a fresh veeam-installation and try to do a FLR to make sure that the restore would basically work. It always worked - until now: Error for every FLR on every vm. I'll create a case to see whats going on here, but it just underlines the need for an integrity check!
Are there any plans meanwhile or am I still the only one?
Thanks!
Are there any plans meanwhile or am I still the only one?
Thanks!
-
- Chief Product Officer
- Posts: 31806
- Liked: 7300 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: object storage integrity check
Indeed, as you can see - still the only one in over 3 years
Nevertheless, the good news is this feature is becoming essential regardless. Simply because V12 will open flood gates of using on-prem object storage as primary backup repositories, requiring us to deliver all the same capabilities for them as are available today with regular backup repositories.
Nevertheless, the good news is this feature is becoming essential regardless. Simply because V12 will open flood gates of using on-prem object storage as primary backup repositories, requiring us to deliver all the same capabilities for them as are available today with regular backup repositories.
-
- Veeam Legend
- Posts: 945
- Liked: 221 times
- Joined: Jul 19, 2016 8:39 am
- Full Name: Michael
- Location: Rheintal, Austria
- Contact:
Re: object storage integrity check
Thanks Anton!
-
- Novice
- Posts: 9
- Liked: never
- Joined: Jun 14, 2012 10:36 am
- Contact:
Re: object storage integrity check
I've experienced similar issue with backups being unrecoverable from object storage which shows the need for some sort of health check. Will this be available in V12?
2 separate issues (cases 05649650, 05658257)
2 separate issues (cases 05649650, 05658257)
-
- Product Manager
- Posts: 9847
- Liked: 2607 times
- Joined: May 13, 2017 4:51 pm
- Full Name: Fabian K.
- Location: Switzerland
- Contact:
Re: object storage integrity check
Hi Dave
Yes. Health check for object storage is planned for V12.
Just as a note, a health check alone will not make sure, that the machine or application will work after a restore.
Periodic restore tests should also be performed.
Thanks
Fabian
Yes. Health check for object storage is planned for V12.
Just as a note, a health check alone will not make sure, that the machine or application will work after a restore.
Periodic restore tests should also be performed.
Thanks
Fabian
Product Management Analyst @ Veeam Software
-
- Novice
- Posts: 9
- Liked: never
- Joined: Jun 14, 2012 10:36 am
- Contact:
Re: object storage integrity check
Hi Fabian,
Thanks for the confirmation if the health check for object storage works the same as the validator tool or backup copy health check, then it would have detected both of our issues.
Thanks for the confirmation if the health check for object storage works the same as the validator tool or backup copy health check, then it would have detected both of our issues.
Who is online
Users browsing this forum: No registered users and 11 guests