-
Matt.Sharpe
- Service Provider
- Posts: 241
- Liked: 20 times
- Joined: Mar 29, 2016 3:37 pm
- Full Name: Matt Sharpe
- Contact:
S3/Object Storage Space Efficiency
This is something I've never quite grasped so hoping someone can explain/confirm it. When I've done space calculations with Veeam calculators. It shows object storage along with ReFS/XFS tick boxes for space savings.
I asked a couple of S3 providers if object storage has space saving capabilities similar to XFS/REFS for fast clone. They said they do NOT.
So the question is, what space savings do you get with S3/Object. Do you have a form of block savings/deduplications which means you are getting a cost effective solution for long term retention. Or does it work more similar to standard NTFS storage. So space saving technology etc?
I asked a couple of S3 providers if object storage has space saving capabilities similar to XFS/REFS for fast clone. They said they do NOT.
So the question is, what space savings do you get with S3/Object. Do you have a form of block savings/deduplications which means you are getting a cost effective solution for long term retention. Or does it work more similar to standard NTFS storage. So space saving technology etc?
-
Gostev
- Chief Product Officer
- Posts: 33047
- Liked: 8114 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: S3/Object Storage Space Efficiency
Yes, the storage space savings are exactly the same with object storage and with XFS/ReFS. Block cloning between restore points prevents duplication from occurring on all these storage types, so DEduplication is consequently not required for either of them.
However, if you enable immutability on S3, there will be space consumption overhead comparing to XFS/ReFS. This is what those S3 providers are probably thinking about. The smaller your immutability period is, the closer object storage space consumption will be to ReFS/XFS.
However, if you enable immutability on S3, there will be space consumption overhead comparing to XFS/ReFS. This is what those S3 providers are probably thinking about. The smaller your immutability period is, the closer object storage space consumption will be to ReFS/XFS.
-
Matt.Sharpe
- Service Provider
- Posts: 241
- Liked: 20 times
- Joined: Mar 29, 2016 3:37 pm
- Full Name: Matt Sharpe
- Contact:
Re: S3/Object Storage Space Efficiency
Hi Gostev,
So S3/Object storage uses a form of block cloning, where it will not store the same block twice? Or does it depend on the redundancy used (erasure coding etc) as to whether you actually see the space savings?
So S3/Object storage uses a form of block cloning, where it will not store the same block twice? Or does it depend on the redundancy used (erasure coding etc) as to whether you actually see the space savings?
-
Gostev
- Chief Product Officer
- Posts: 33047
- Liked: 8114 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: S3/Object Storage Space Efficiency
It is the former.
-
Matt.Sharpe
- Service Provider
- Posts: 241
- Liked: 20 times
- Joined: Mar 29, 2016 3:37 pm
- Full Name: Matt Sharpe
- Contact:
Re: S3/Object Storage Space Efficiency
Thanks Gostev, do you know if there is any documentation of white paper on this?
-
Gostev
- Chief Product Officer
- Posts: 33047
- Liked: 8114 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: S3/Object Storage Space Efficiency
Sure, for example What's New in V12 document specifically calls this similarity out.
-
mkretzer
- Veeam Legend
- Posts: 1321
- Liked: 474 times
- Joined: Dec 17, 2015 7:17 am
- Contact:
Re: S3/Object Storage Space Efficiency
According to V13 calculators (https://www.veeam.com/calculators/simple/vbr/machines) direct backup to S3 is now more space efficient as XFS. Is that really the case? How does that work?
-
Gostev
- Chief Product Officer
- Posts: 33047
- Liked: 8114 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: S3/Object Storage Space Efficiency
It's not correct, direct backup to S3 is not more space efficient than XFS.
-
mkretzer
- Veeam Legend
- Posts: 1321
- Liked: 474 times
- Joined: Dec 17, 2015 7:17 am
- Contact:
Re: S3/Object Storage Space Efficiency
Ok - did i use the calculator wrong?
I only switch from XFS to S3 and (without putting in more data) it goes from 1,5 TB down to 1,1 TB.
Can you estimate how much more space is needed when using S3 instead of XFS/ReFS (i know its dependent on the data but i need a rough estimate).
Markus
I only switch from XFS to S3 and (without putting in more data) it goes from 1,5 TB down to 1,1 TB.
Can you estimate how much more space is needed when using S3 instead of XFS/ReFS (i know its dependent on the data but i need a rough estimate).
Markus
-
Gostev
- Chief Product Officer
- Posts: 33047
- Liked: 8114 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: S3/Object Storage Space Efficiency
No, it's just that the calculator can be very wonky.
But it's not a product of Veeam R&D so we can't comment on the results or provide our own disk space estimations.
You can discuss this questions directly with the Calculator maintainers on the Community Hub at
https://community.veeam.com/groups/calc ... ommons-151
But it's not a product of Veeam R&D so we can't comment on the results or provide our own disk space estimations.
You can discuss this questions directly with the Calculator maintainers on the Community Hub at
https://community.veeam.com/groups/calc ... ommons-151
-
mkretzer
- Veeam Legend
- Posts: 1321
- Liked: 474 times
- Joined: Dec 17, 2015 7:17 am
- Contact:
Re: S3/Object Storage Space Efficiency
Will do - what about the estimation how much more space is needed - does R&D have any experience about that?
-
Gostev
- Chief Product Officer
- Posts: 33047
- Liked: 8114 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: S3/Object Storage Space Efficiency
We don't, as I've said this is handled by a different team that manages all related knowledge. They are field folks (solution architects) who interact with real-world environment on a day to day basis and thus not limited to theory only.
-
mame17
- Lurker
- Posts: 1
- Liked: never
- Joined: Dec 23, 2025 7:17 am
- Full Name: mamequin
- Contact:
Re: S3/Object Storage Space Efficiency
You’re basically circling the right question, and the confusion is very common because S3 savings don’t come from the storage platform in the same way ReFS/XFS savings do.
Short answer: S3/Object storage itself does not do block cloning like ReFS/XFS. The space efficiency you see in Veeam calculators comes from how Veeam writes data to object storage, not from native S3 magic.
Here’s the clearer breakdown:
ReFS/XFS
True block cloning at the filesystem level
New restore points reference existing blocks
Near-zero growth between incrementals
Storage system understands blocks and pointers
S3/Object Storage
No block-level awareness
No native fast clone
No cross-object deduplication at the provider level
Each object is immutable once written
So where do the “savings” come from?
They come from Veeam’s object storage format, not S3 itself:
Veeam splits backups into small extents/objects
Identical data segments are logically referenced by metadata
Incrementals don’t resend unchanged data
This works regardless of erasure coding or replication redundancy only affects physical footprint, not logical efficiency
Immutability is the big difference
With S3 Object Lock, old objects cannot be reused or overwritten
This prevents Veeam from “collapsing” chains as efficiently as ReFS/XFS
Longer immutability = more metadata + more retained objects = more overhead
That’s why providers often say “no space savings” they’re thinking storage-side, not application-side
Real-world example
I’ve seen a repo where:
ReFS local repo grew ~1–2% per day
S3 immutable repo grew ~5–8% per day with 30-day immutability
Same backup job, same data different mechanics.
Think of it like using a Couple Finance Tracker,https://play.google.com/store/apps/deta ... gettracker you’re not duplicating every expense each month, you’re referencing past data intelligently — but if you lock past months and can’t adjust them, you’ll naturally accumulate more records over time.
Bottom line
S3 ≠ block cloning
Savings are application-driven, not storage-driven
Erasure coding doesn’t affect logical savings
Immutability introduces unavoidable overhead
Still very cost-effective for long-term retention and ransomware protection
That’s why Veeam calculators show savings but they’re not the same kind of savings you get from ReFS/XFS fast clone.
Short answer: S3/Object storage itself does not do block cloning like ReFS/XFS. The space efficiency you see in Veeam calculators comes from how Veeam writes data to object storage, not from native S3 magic.
Here’s the clearer breakdown:
ReFS/XFS
True block cloning at the filesystem level
New restore points reference existing blocks
Near-zero growth between incrementals
Storage system understands blocks and pointers
S3/Object Storage
No block-level awareness
No native fast clone
No cross-object deduplication at the provider level
Each object is immutable once written
So where do the “savings” come from?
They come from Veeam’s object storage format, not S3 itself:
Veeam splits backups into small extents/objects
Identical data segments are logically referenced by metadata
Incrementals don’t resend unchanged data
This works regardless of erasure coding or replication redundancy only affects physical footprint, not logical efficiency
Immutability is the big difference
With S3 Object Lock, old objects cannot be reused or overwritten
This prevents Veeam from “collapsing” chains as efficiently as ReFS/XFS
Longer immutability = more metadata + more retained objects = more overhead
That’s why providers often say “no space savings” they’re thinking storage-side, not application-side
Real-world example
I’ve seen a repo where:
ReFS local repo grew ~1–2% per day
S3 immutable repo grew ~5–8% per day with 30-day immutability
Same backup job, same data different mechanics.
Think of it like using a Couple Finance Tracker,https://play.google.com/store/apps/deta ... gettracker you’re not duplicating every expense each month, you’re referencing past data intelligently — but if you lock past months and can’t adjust them, you’ll naturally accumulate more records over time.
Bottom line
S3 ≠ block cloning
Savings are application-driven, not storage-driven
Erasure coding doesn’t affect logical savings
Immutability introduces unavoidable overhead
Still very cost-effective for long-term retention and ransomware protection
That’s why Veeam calculators show savings but they’re not the same kind of savings you get from ReFS/XFS fast clone.
-
Gostev
- Chief Product Officer
- Posts: 33047
- Liked: 8114 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: S3/Object Storage Space Efficiency
The post above provides an exceptionally good technical explanation of the "magic" behind space savings and where they come from in case of file and object storage. Thank you so much for your time putting it together!
However, a few important notes:
1/ Comparing actual disk space consumption between non-immutable repo (ReFS) with an immutable repo (S3) is not very fair. A more fair comparison would be ReFS with non-immutable S3, or immutable XFS with immutable S3.
2/ If you compare the latter, you would still find that in V12, noticeably more space (easily 2x) will be consumed in the immutable S3 repository comparing to the immutable XFS one. This situation has been significantly improved in V13 because we changed the mechanics of storing backups in S3 (what you called "Veeam’s object storage format")
- In version 13.0, new mechanics is available when backing up directly to object storage (including Backup Copy jobs)
- Version 13.1 will bring the new mechanics also to SOBR Capacity Tier
Since V13 is so fresh, we don't yet have much real-world data comparing the storage efficiency between immutable XFS and immutable S3 similar to what you have shared above. Having said that, I expect XFS to still be more efficient as it comes to disk space consumption, just no longer by a mile like with V12.
However, a few important notes:
1/ Comparing actual disk space consumption between non-immutable repo (ReFS) with an immutable repo (S3) is not very fair. A more fair comparison would be ReFS with non-immutable S3, or immutable XFS with immutable S3.
2/ If you compare the latter, you would still find that in V12, noticeably more space (easily 2x) will be consumed in the immutable S3 repository comparing to the immutable XFS one. This situation has been significantly improved in V13 because we changed the mechanics of storing backups in S3 (what you called "Veeam’s object storage format")
- In version 13.0, new mechanics is available when backing up directly to object storage (including Backup Copy jobs)
- Version 13.1 will bring the new mechanics also to SOBR Capacity Tier
Since V13 is so fresh, we don't yet have much real-world data comparing the storage efficiency between immutable XFS and immutable S3 similar to what you have shared above. Having said that, I expect XFS to still be more efficient as it comes to disk space consumption, just no longer by a mile like with V12.
Who is online
Users browsing this forum: No registered users and 2 guests