Discussions related to using object storage as a backup target.
Post Reply
Matt.Sharpe
Service Provider
Posts: 241
Liked: 20 times
Joined: Mar 29, 2016 3:37 pm
Full Name: Matt Sharpe
Contact:

S3/Object Storage Space Efficiency

Post by Matt.Sharpe »

This is something I've never quite grasped so hoping someone can explain/confirm it. When I've done space calculations with Veeam calculators. It shows object storage along with ReFS/XFS tick boxes for space savings.

I asked a couple of S3 providers if object storage has space saving capabilities similar to XFS/REFS for fast clone. They said they do NOT.

So the question is, what space savings do you get with S3/Object. Do you have a form of block savings/deduplications which means you are getting a cost effective solution for long term retention. Or does it work more similar to standard NTFS storage. So space saving technology etc?
Gostev
Chief Product Officer
Posts: 33047
Liked: 8114 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: S3/Object Storage Space Efficiency

Post by Gostev »

Yes, the storage space savings are exactly the same with object storage and with XFS/ReFS. Block cloning between restore points prevents duplication from occurring on all these storage types, so DEduplication is consequently not required for either of them.

However, if you enable immutability on S3, there will be space consumption overhead comparing to XFS/ReFS. This is what those S3 providers are probably thinking about. The smaller your immutability period is, the closer object storage space consumption will be to ReFS/XFS.
Matt.Sharpe
Service Provider
Posts: 241
Liked: 20 times
Joined: Mar 29, 2016 3:37 pm
Full Name: Matt Sharpe
Contact:

Re: S3/Object Storage Space Efficiency

Post by Matt.Sharpe »

Hi Gostev,

So S3/Object storage uses a form of block cloning, where it will not store the same block twice? Or does it depend on the redundancy used (erasure coding etc) as to whether you actually see the space savings?
Gostev
Chief Product Officer
Posts: 33047
Liked: 8114 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: S3/Object Storage Space Efficiency

Post by Gostev »

It is the former.
Matt.Sharpe
Service Provider
Posts: 241
Liked: 20 times
Joined: Mar 29, 2016 3:37 pm
Full Name: Matt Sharpe
Contact:

Re: S3/Object Storage Space Efficiency

Post by Matt.Sharpe »

Thanks Gostev, do you know if there is any documentation of white paper on this?
Gostev
Chief Product Officer
Posts: 33047
Liked: 8114 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: S3/Object Storage Space Efficiency

Post by Gostev »

Sure, for example What's New in V12 document specifically calls this similarity out.
mkretzer
Veeam Legend
Posts: 1321
Liked: 474 times
Joined: Dec 17, 2015 7:17 am
Contact:

Re: S3/Object Storage Space Efficiency

Post by mkretzer »

According to V13 calculators (https://www.veeam.com/calculators/simple/vbr/machines) direct backup to S3 is now more space efficient as XFS. Is that really the case? How does that work?
Gostev
Chief Product Officer
Posts: 33047
Liked: 8114 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: S3/Object Storage Space Efficiency

Post by Gostev »

It's not correct, direct backup to S3 is not more space efficient than XFS.
mkretzer
Veeam Legend
Posts: 1321
Liked: 474 times
Joined: Dec 17, 2015 7:17 am
Contact:

Re: S3/Object Storage Space Efficiency

Post by mkretzer »

Ok - did i use the calculator wrong?
I only switch from XFS to S3 and (without putting in more data) it goes from 1,5 TB down to 1,1 TB.

Can you estimate how much more space is needed when using S3 instead of XFS/ReFS (i know its dependent on the data but i need a rough estimate).

Markus
Gostev
Chief Product Officer
Posts: 33047
Liked: 8114 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: S3/Object Storage Space Efficiency

Post by Gostev »

No, it's just that the calculator can be very wonky.

But it's not a product of Veeam R&D so we can't comment on the results or provide our own disk space estimations.

You can discuss this questions directly with the Calculator maintainers on the Community Hub at
https://community.veeam.com/groups/calc ... ommons-151
mkretzer
Veeam Legend
Posts: 1321
Liked: 474 times
Joined: Dec 17, 2015 7:17 am
Contact:

Re: S3/Object Storage Space Efficiency

Post by mkretzer »

Will do - what about the estimation how much more space is needed - does R&D have any experience about that?
Gostev
Chief Product Officer
Posts: 33047
Liked: 8114 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: S3/Object Storage Space Efficiency

Post by Gostev »

We don't, as I've said this is handled by a different team that manages all related knowledge. They are field folks (solution architects) who interact with real-world environment on a day to day basis and thus not limited to theory only.
mame17
Lurker
Posts: 1
Liked: never
Joined: Dec 23, 2025 7:17 am
Full Name: mamequin
Contact:

Re: S3/Object Storage Space Efficiency

Post by mame17 »

You’re basically circling the right question, and the confusion is very common because S3 savings don’t come from the storage platform in the same way ReFS/XFS savings do.

Short answer: S3/Object storage itself does not do block cloning like ReFS/XFS. The space efficiency you see in Veeam calculators comes from how Veeam writes data to object storage, not from native S3 magic.

Here’s the clearer breakdown:

ReFS/XFS

True block cloning at the filesystem level

New restore points reference existing blocks

Near-zero growth between incrementals

Storage system understands blocks and pointers

S3/Object Storage

No block-level awareness

No native fast clone

No cross-object deduplication at the provider level

Each object is immutable once written

So where do the “savings” come from?

They come from Veeam’s object storage format, not S3 itself:

Veeam splits backups into small extents/objects

Identical data segments are logically referenced by metadata

Incrementals don’t resend unchanged data

This works regardless of erasure coding or replication redundancy only affects physical footprint, not logical efficiency

Immutability is the big difference

With S3 Object Lock, old objects cannot be reused or overwritten

This prevents Veeam from “collapsing” chains as efficiently as ReFS/XFS

Longer immutability = more metadata + more retained objects = more overhead

That’s why providers often say “no space savings” they’re thinking storage-side, not application-side

Real-world example
I’ve seen a repo where:

ReFS local repo grew ~1–2% per day

S3 immutable repo grew ~5–8% per day with 30-day immutability
Same backup job, same data different mechanics.

Think of it like using a Couple Finance Tracker,https://play.google.com/store/apps/deta ... gettracker you’re not duplicating every expense each month, you’re referencing past data intelligently — but if you lock past months and can’t adjust them, you’ll naturally accumulate more records over time.

Bottom line

S3 ≠ block cloning

Savings are application-driven, not storage-driven

Erasure coding doesn’t affect logical savings

Immutability introduces unavoidable overhead

Still very cost-effective for long-term retention and ransomware protection

That’s why Veeam calculators show savings but they’re not the same kind of savings you get from ReFS/XFS fast clone.
Gostev
Chief Product Officer
Posts: 33047
Liked: 8114 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: S3/Object Storage Space Efficiency

Post by Gostev »

The post above provides an exceptionally good technical explanation of the "magic" behind space savings and where they come from in case of file and object storage. Thank you so much for your time putting it together!

However, a few important notes:

1/ Comparing actual disk space consumption between non-immutable repo (ReFS) with an immutable repo (S3) is not very fair. A more fair comparison would be ReFS with non-immutable S3, or immutable XFS with immutable S3.

2/ If you compare the latter, you would still find that in V12, noticeably more space (easily 2x) will be consumed in the immutable S3 repository comparing to the immutable XFS one. This situation has been significantly improved in V13 because we changed the mechanics of storing backups in S3 (what you called "Veeam’s object storage format")
- In version 13.0, new mechanics is available when backing up directly to object storage (including Backup Copy jobs)
- Version 13.1 will bring the new mechanics also to SOBR Capacity Tier

Since V13 is so fresh, we don't yet have much real-world data comparing the storage efficiency between immutable XFS and immutable S3 similar to what you have shared above. Having said that, I expect XFS to still be more efficient as it comes to disk space consumption, just no longer by a mile like with V12.
Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests