Discussions related to using object storage as a backup target.
Locked
mkh
Service Provider
Posts: 64
Liked: 18 times
Joined: Apr 20, 2018 6:17 am
Full Name: Michael Høyer
Contact:

VBR v12 object storage format change, will that mean higher data usage?

Post by mkh » 1 person likes this post

I have just watched this video https://anthonyspiteri.net/v12-preview- ... ancements/
and i get i little unsure about what exactly these changes to the object storage format means for the usage

In the video Anthony and Dustin talks about reuse of blocks now only being for neighboring checkpoints inside 3 days, as part of removing the index

But what does that means if you do a synthetic full every saturday, will there not be any reuse between these?

Taken at "face value" what Anthony and Dustin is saying will mean that storage usage on object storage will explode in V12 compared to V11 if you are for instance keeping 12 monthlies and 5 yearly full.

I hope i'm missing something here :)
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by veremin »

I'm yet to see the session, but in the provided example only blocks that have not been already present on the object storage repository will be written.

However, at this moment we cannot comment on yet unreleased features or discuss any specifics as some things are still subject to possible changes in the final code.

Thank for understanding!
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by Gostev »

V12 will still reuse unchanged disk blocks even from a 5 years old restore point. I don't have time to watch the recording but if people understand from whatever is said there that "reuse is only within 3 days" then we need to pull that part of the video anyway as it's not correct.

@veremin please coordinate this activity with Dustin and Anthony.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by veremin »

Sure, will discuss it internally.
TonioRoffo
Enthusiast
Posts: 52
Liked: 5 times
Joined: Jun 18, 2009 2:27 pm
Full Name: Yves Smolders
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by TonioRoffo »

Is block reuse also valid for direct to object storage backups, or only in SOBR?

Thanks
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by Gostev » 1 person likes this post

It's the same new format in both.
codylittle
Influencer
Posts: 21
Liked: 1 time
Joined: Jul 06, 2022 3:52 am
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by codylittle »

Gostev wrote: Jun 20, 2022 5:59 pm V12 will still reuse unchanged disk blocks even from a 5 years old restore point.
Whilst participating within the V12 beta, we have not observed this in our testing. Each weekly full is being uploaded in full.
During the VeeamON Forum ANZ (Deep Dive of V12: Object Storage Enhancements VOD 14:30) it was mentioned that Veeam will only reuse blocks amongst neighbouring GFS checkpoints.
Currently working with our presales SE around this, but just wondering if anyone else can shed some light on this?
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by Gostev »

Yes, please submit beta logs to the presales engineer who provided you the beta for investigation by R&D, as this is not an expected behavior and we did not have any other beta participants experience this.

I have asked to the review the recording if it provides incorrect information, or has a bad explanation, or if perhaps it is just your misinterpretation that makes you think this approach somehow prevents what I said in the quoted statement. Depending on the result we will keep or delete that video.
codylittle
Influencer
Posts: 21
Liked: 1 time
Joined: Jul 06, 2022 3:52 am
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by codylittle »

I'm fairly confident I'm probably misinterpreting what's meant 😅
I've pm'd you a link to the VOD
TonioRoffo
Enthusiast
Posts: 52
Liked: 5 times
Joined: Jun 18, 2009 2:27 pm
Full Name: Yves Smolders
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by TonioRoffo »

codylittle wrote: Sep 29, 2022 11:25 am Whilst participating within the V12 beta, we have not observed this in our testing. Each weekly full is being uploaded in full.
I hope that is not the case, that would negate our move to S3/Wasabi for all our clients.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by Gostev » 2 people like this post

It's not the case.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by Gostev » 1 person likes this post

codylittle wrote: Sep 29, 2022 11:25 amDuring the VeeamON Forum ANZ (Deep Dive of V12: Object Storage Enhancements VOD 14:30) it was mentioned that Veeam will only reuse blocks amongst neighbouring GFS checkpoints.
@veremin has watched the presentation and confirmed that it provides invalid information. It appears that the presenter took the [already known] invalid source of information (the one that started this thread, we've been playing "whack a mole" with it ever since) and further added his own mistakes on top due to lack of understanding. This recording will be deleted.
codylittle
Influencer
Posts: 21
Liked: 1 time
Joined: Jul 06, 2022 3:52 am
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by codylittle » 1 person likes this post

Thanks for the confirmation regarding this Gostev, has definitely been a point of worry.
We'll continue working with our presales SE to work out why all our weeklies are being uploaded in full.
TonioRoffo
Enthusiast
Posts: 52
Liked: 5 times
Joined: Jun 18, 2009 2:27 pm
Full Name: Yves Smolders
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by TonioRoffo »

Thank you for the updates, now we know we don't have to worry about this.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by veremin »

The video has been removed and will not cause further confusion. Thanks for raising this issue; much appreciated.
TonioRoffo
Enthusiast
Posts: 52
Liked: 5 times
Joined: Jun 18, 2009 2:27 pm
Full Name: Yves Smolders
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by TonioRoffo » 2 people like this post

So,

Just to be sure:

* Under V12, we can schedule a job to go direct to S3 storage, and use GFS to retain weeklies, monthlies, ... without consuming "full backups" at every GFS point?

Thanks
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by Gostev » 2 people like this post

That is correct.
TonioRoffo
Enthusiast
Posts: 52
Liked: 5 times
Joined: Jun 18, 2009 2:27 pm
Full Name: Yves Smolders
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by TonioRoffo »

Hello, Gostev, this doesn't seem to be the case in my first test:

Image

Backup at 12:24 was a full, as expected.
12:28 was an incremental, also as expected
12:30 was an active full - adds an entire 947MB to the S3 storage
12:44 was triggered by a GFS setting to keep a weekly full (in SOBR, this would trigger a synthetic backup and hardly take any place) -> **adds an entire 947MB**

This is not what I expected to happen. That also means that, using local S3 repositories (minio), like in this example, is no alternative for ReFS or Linux XFS local repostitories!?

I expected the GFS backup not to reuse blocks which are already stored on the S3 storage.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VBR v12 object storage format change, will that mean higher data usage?

Post by Gostev » 1 person likes this post

Actually, ReFS/XFS repository would show you exact the same number in this dialog, because this value is just a sum of logical restore point sizes. The dialog simply does not have a capability to reach out to the storage device and query the actual physical disk space consumption, which will always be lower than the number displayed in the dialog due to the block cloning.

I'm locking this thread because it has wrong information in the first post which keeps being bumped. I believe all questions and concerns regarding said invalid information have been answered in the greatest of details already.

If you have objective metrics (physical bucket size) indicating that block cloning is not being leveraged, then this is not an expected behavior or a known issue. If anyone is experiencing this, please open a support case to investigate the reason.
Locked

Who is online

Users browsing this forum: JasonBailey, tkado and 7 guests