-
- Influencer
- Posts: 13
- Liked: never
- Joined: Jun 26, 2023 6:32 am
- Full Name: arthur
- Contact:
How to Reduce Block Generation Days
Hello,
We are currently using VBR v12 to backup data to Object Repository with immutability feature.
We noticed that an extra 10 days called Block Generation is added to the immutability period.
object-storage-f52/aws-s3-how-to-reduce ... 68858.html
From this post it mentioned that in VBR v11, you can add a registry key named SOBRCapacityImmutabilityGenerationDays to control the block generation days, but it seems invalid in VBR v12.
My question:
How can we reduce the block generation days in VBR v12 if we try to backup following different modes?
- Using SOBR with Object Repository (immutability on)
- Directly to Object Repository (immutability on)
Thanks
Arthur
We are currently using VBR v12 to backup data to Object Repository with immutability feature.
We noticed that an extra 10 days called Block Generation is added to the immutability period.
object-storage-f52/aws-s3-how-to-reduce ... 68858.html
From this post it mentioned that in VBR v11, you can add a registry key named SOBRCapacityImmutabilityGenerationDays to control the block generation days, but it seems invalid in VBR v12.
My question:
How can we reduce the block generation days in VBR v12 if we try to backup following different modes?
- Using SOBR with Object Repository (immutability on)
- Directly to Object Repository (immutability on)
Thanks
Arthur
-
- Product Manager
- Posts: 14953
- Liked: 3149 times
- Joined: Sep 01, 2014 11:46 am
- Full Name: Hannes Kasparick
- Location: Austria
- Contact:
Re: How to Reduce Block Generation Days
Hello,
and welcome to the forums.
ObjectStorageImmutabilityGenerationDays is the V12 key. But please remember... reducing the value will lead to more API calls = more IO operation and more costs if your cloud provider charges for API calls.
Best regards,
Hannes
and welcome to the forums.
ObjectStorageImmutabilityGenerationDays is the V12 key. But please remember... reducing the value will lead to more API calls = more IO operation and more costs if your cloud provider charges for API calls.
Best regards,
Hannes
-
- Influencer
- Posts: 13
- Liked: never
- Joined: Jun 26, 2023 6:32 am
- Full Name: arthur
- Contact:
Re: How to Reduce Block Generation Days
Hello Hannes,
Thank you for the reply.
We added a DWORD named ObjectStorageImmutabilityGenerationDays in HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication, and the value is set to 0. We rebooted the VBR server, but after a full backup to object repository, the block generation days remain unchanged (default 10).
Did we follow the correct setting procedure?
Regards,
Arthur
Thank you for the reply.
We added a DWORD named ObjectStorageImmutabilityGenerationDays in HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication, and the value is set to 0. We rebooted the VBR server, but after a full backup to object repository, the block generation days remain unchanged (default 10).
Did we follow the correct setting procedure?
Regards,
Arthur
-
- Product Manager
- Posts: 14953
- Liked: 3149 times
- Joined: Sep 01, 2014 11:46 am
- Full Name: Hannes Kasparick
- Location: Austria
- Contact:
Re: How to Reduce Block Generation Days
Hello,
reboot / restart of the service is not needed. 1 is the lowest value that makes sense. If that does not work, I can only suggest to check with support.
Best regards,
Hannes
reboot / restart of the service is not needed. 1 is the lowest value that makes sense. If that does not work, I can only suggest to check with support.
Best regards,
Hannes
-
- VP, Product Management
- Posts: 7145
- Liked: 1531 times
- Joined: May 04, 2011 8:36 am
- Full Name: Andreas Neufert
- Location: Germany
- Contact:
Re: How to Reduce Block Generation Days
I strongly advice against this change. A lot of functionality depends on this and all kind of optimizations are done in that way.
Remember the 10 extra days are up to 10 extra days and over time (after many restore points) you will likely have less than 10% additional data protected form some days longer.
On the other side as Hannes said you do not force us update huge amount of your objects at any backup run with additional immutability settings.
The concept is not that simple to understand and depend heavily on your change rate and what data is overwritten.
Remember as well that we do not store multiple fulls in the backup. We store blocks as objects. So the block generation in the backend is like a filesystem housekeeping for immutability blocks.
Why do you want to change this in the first place?
Remember the 10 extra days are up to 10 extra days and over time (after many restore points) you will likely have less than 10% additional data protected form some days longer.
On the other side as Hannes said you do not force us update huge amount of your objects at any backup run with additional immutability settings.
The concept is not that simple to understand and depend heavily on your change rate and what data is overwritten.
Remember as well that we do not store multiple fulls in the backup. We store blocks as objects. So the block generation in the backend is like a filesystem housekeeping for immutability blocks.
Why do you want to change this in the first place?
-
- Influencer
- Posts: 13
- Liked: never
- Joined: Jun 26, 2023 6:32 am
- Full Name: arthur
- Contact:
Re: How to Reduce Block Generation Days
Hello Andreas,
Thank you for the explanation.
We wanted to change the block generation days only because we want the actual immutation period to be exactly the same as what we define in VBR Object Repository.
We understand the block generation is a background optimization especially when we need to do incremental backup. But still we don't expect it will affect the actual immutation period.
In other words, if we want 10 days immutability period, we will set the period to 10 days, but we always get 20 days.
Thanks,
Arthur
Thank you for the explanation.
We wanted to change the block generation days only because we want the actual immutation period to be exactly the same as what we define in VBR Object Repository.
We understand the block generation is a background optimization especially when we need to do incremental backup. But still we don't expect it will affect the actual immutation period.
In other words, if we want 10 days immutability period, we will set the period to 10 days, but we always get 20 days.
Thanks,
Arthur
-
- Product Manager
- Posts: 14953
- Liked: 3149 times
- Joined: Sep 01, 2014 11:46 am
- Full Name: Hannes Kasparick
- Location: Austria
- Contact:
Re: How to Reduce Block Generation Days
just be aware, that it might overload your object storage or result in significant API costs. or the cloud provider throttles you with hard-to-predict side-effects
the reason you mention does not justify using the reg key from my side. the downsides are too big.
the reason you mention does not justify using the reg key from my side. the downsides are too big.
-
- Influencer
- Posts: 13
- Liked: never
- Joined: Jun 26, 2023 6:32 am
- Full Name: arthur
- Contact:
Re: How to Reduce Block Generation Days
OK, we decide to remove the registry key now. Thanks for the advice.
-
- VP, Product Management
- Posts: 7145
- Liked: 1531 times
- Joined: May 04, 2011 8:36 am
- Full Name: Andreas Neufert
- Location: Germany
- Contact:
Re: How to Reduce Block Generation Days
This is a misconception. Block generation will NOT add additional immutable days on your backups. However some of the blocks are marked longer immutable to optimize IO on the storage in the background. And it does not mean it is 10 days. It is in between 0 and 10 days in reality. Again this does not mean that you have additional restore points immutable. Just some background optimization that treat in some extra space to reduce IO on the storage. This is mainly for reducing cost at cloud object storage that charge for IO but it will help as well to save money on the on-premises storage as you need less nodes to cover the IO demand.
-
- Veteran
- Posts: 606
- Liked: 89 times
- Joined: Dec 20, 2015 6:24 pm
- Contact:
Re: How to Reduce Block Generation Days
I can not confirm this. We have a retention of 70 day and immutability of 14 days (we really want 70 to have it the same time as retention, but support recommended to keep it not longer than 14 days). Yet the backups are way longer immutable than the 70 days. Sometimes 100-120 days. This was discussed multiple times and it was always pointed to block generations, even from support. It it not predictable for us when we can delete an orpahned backup as immutability is prononged over and over.
-
- VP, Product Management
- Posts: 7145
- Liked: 1531 times
- Joined: May 04, 2011 8:36 am
- Full Name: Andreas Neufert
- Location: Germany
- Contact:
Re: How to Reduce Block Generation Days
Hello Pirx, if you had settings in the past with longer retention then these backups are marked and stay marked for this specific amount of time. We do not renew retention to blocks that belong only to older backups. What is the support case number you are referring to?
-
- VP, Product Management
- Posts: 7145
- Liked: 1531 times
- Joined: May 04, 2011 8:36 am
- Full Name: Andreas Neufert
- Location: Germany
- Contact:
Re: How to Reduce Block Generation Days
Is it 05407008 or is there a newer one?
-
- Veteran
- Posts: 606
- Liked: 89 times
- Joined: Dec 20, 2015 6:24 pm
- Contact:
Re: How to Reduce Block Generation Days
Neither retention nor immutability time has changed in past 1-2 years. Cases are quite old and there are also old discussions about this in the forum. At some point we just accepted it. Will check case and let you know.
-
- VP, Product Management
- Posts: 7145
- Liked: 1531 times
- Joined: May 04, 2011 8:36 am
- Full Name: Andreas Neufert
- Location: Germany
- Contact:
Re: How to Reduce Block Generation Days
I am sorry to hear that it does not work as for anyone else. If you share the number I can investigate and check. Will follow up with Hannes from above and your Field SE as well then.
-
- Expert
- Posts: 110
- Liked: 11 times
- Joined: Aug 27, 2021 12:29 am
- Contact:
Re: How to Reduce Block Generation Days
10 days make us confusing. So, what is the exactly immutable days?
Andreas Neufert wrote: ↑Jun 28, 2023 10:06 am This is a misconception. Block generation will NOT add additional immutable days on your backups. However some of the blocks are marked longer immutable to optimize IO on the storage in the background. And it does not mean it is 10 days. It is in between 0 and 10 days in reality. Again this does not mean that you have additional restore points immutable. Just some background optimization that treat in some extra space to reduce IO on the storage. This is mainly for reducing cost at cloud object storage that charge for IO but it will help as well to save money on the on-premises storage as you need less nodes to cover the IO demand.
-
- Product Manager
- Posts: 14953
- Liked: 3149 times
- Joined: Sep 01, 2014 11:46 am
- Full Name: Hannes Kasparick
- Location: Austria
- Contact:
Re: How to Reduce Block Generation Days
Hello,
the guaranteed immutability is the time you configure. The "exact" immutable time is the "configured time" + "1-10 days" depending. Without going too much into detail: you probably don't want to change that value.
the user guide has more information https://helpcenter.veeam.com/docs/backu ... ml?ver=120
Best regards,
Hannes
the guaranteed immutability is the time you configure. The "exact" immutable time is the "configured time" + "1-10 days" depending. Without going too much into detail: you probably don't want to change that value.
the user guide has more information https://helpcenter.veeam.com/docs/backu ... ml?ver=120
Best regards,
Hannes
-
- VP, Product Management
- Posts: 7145
- Liked: 1531 times
- Joined: May 04, 2011 8:36 am
- Full Name: Andreas Neufert
- Location: Germany
- Contact:
Re: How to Reduce Block Generation Days
As we do incremental forever processing objects in the cloud will over time belong to newer restore points. Without the block generation days, on a daily base you would need to change the majority (97% of a full) of the objects immutability value to +1 day which is IO intensive (load on the object storage) and costly (if you do this in public clouds that charge for any S3 operation). Our block generation is basically a mathematical prediction to set for those objects that we think are needed longer (as they will be part of the future restore points) an a bit longer immutability date, so that we do not have to update the immutability value for them frequently.
Again this is a internal background process that no one should care and not change (it just works and optimizes things for you). Set the immutability to the amount of days you need it and you are done.
Again this is a internal background process that no one should care and not change (it just works and optimizes things for you). Set the immutability to the amount of days you need it and you are done.
-
- Lurker
- Posts: 1
- Liked: never
- Joined: Mar 02, 2021 9:43 am
- Contact:
Re: How to Reduce Block Generation Days
Hi there,
as a service provider with its own on-prem S3 stroage we were also thinking about lowering the block generation days, since we don't have API costs. I understand that you still not recommend it because of some internal processing and IO load, so could you explain what actully happens in the following scenario:
- Backup job with 14 days retention
- Backup repository with 14 days immutability.
So the first full backup will be kept for 24 days (10 days block generation). What exactly would happen in the future? How many restore points will be kept in the long run?
Does this configuration even make sense to you? Or what is your immutability reccommendation for a Backup Job with 14 days rentention and a Backup Job with 30 days retention?
Thanks and kind regards
Tobias
as a service provider with its own on-prem S3 stroage we were also thinking about lowering the block generation days, since we don't have API costs. I understand that you still not recommend it because of some internal processing and IO load, so could you explain what actully happens in the following scenario:
- Backup job with 14 days retention
- Backup repository with 14 days immutability.
So the first full backup will be kept for 24 days (10 days block generation). What exactly would happen in the future? How many restore points will be kept in the long run?
Does this configuration even make sense to you? Or what is your immutability reccommendation for a Backup Job with 14 days rentention and a Backup Job with 30 days retention?
Thanks and kind regards
Tobias
-
- Product Manager
- Posts: 20465
- Liked: 2327 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: How to Reduce Block Generation Days
A few months ago we updated the section in our User Guide that covers immutability operations. It explains how immutability settings and generation periods influence the actual retention, determining the number of points or objects present on your storage system. I recommend familiarizing yourself with it (if you haven't already) and returning with any clarifying questions, if there are any remaining.
Thanks!
Thanks!
Who is online
Users browsing this forum: mlkk and 17 guests