[Feature Request] Allow SOSAPI enabled S3 Storage to Adjust Block Sizes on Jobs targeting S3 Storage

[tyler.jurgens]

Many on-premises object storage implementations request a larger block size for many reasons, which I won't go into here but include Scalability, Performance, and Resiliency. However, Veeam, by default always implements a 1 MB block size.

This makes it difficult for service providers to offer their own solutions with Veeam. Other vendors handle block sizes significantly differently than Veeam does, so this problem becomes a uniquely Veeam one (at least, from my perspective with the different Vendors we've dealt with). Whether Veeam's implementation is better or worse is also not the point of this FR, I only state it as a reality of what we do and the challenges we face.

However, Veeam has this fantastic feature they created called the SOSAPI that can help here! The SOSAPI implemented by a compatible S3 vendor can advise Veeam on which block size it wants to see in order to work best on its platform. My request is that Veeam expand upon this implementation and allow the SOSAPI-integrated S3 to set the block size on all Veeam jobs targeting that S3 bucket. I would envision this as all new jobs targeting the S3 bucket having that block size set automatically. Other jobs that may already target that S3 bucket, but pre-date this FR being in production, have the block size "recommendation" set to the value returned by the SOSAPI. That way all new jobs are set appropriately, while older jobs need manual intervention (adjust the block size, run an active full).

I understand increasing the block size will also increase the backup size by 50% over the setting before it (4 MB blocks take 50% more storage than 1 MB blocks, and 8 MB blocks that 50% more storage than 4 MB blocks), but if *only* the jobs targeting the S3 buckets are affected, it minimizes the effects on the entire Veeam environment to only the S3 jobs targeting only that S3 bucket. Furthermore, Veeam has been pushing S3 as the way of the future, but many service providers simply will not resell a hyper-scaler's solution. Frankly, it's not good for the entire IT ecosystem to be swallowed up by a handful of large entities so there needs to be a choice. Veeam can help ensure customers have options.

Just so Veeam doesn't feel all the pressure, I have also been pushing our S3 vendors to improve their performance under Veeam workloads. Everyone has a part to play here.

I only know of one on-premises S3 vendor that is fine with 1 MB blocks, but they do not have any erasure coding, nor do they scale to the size we would need to be able to implement them on our side. Even if they solved the scale, the lack of erasure coding leaves something to be desired in terms of the resiliency of the data.

This Feature Request can go hand in hand with the below feature request but could be implemented separately. That said, it would make the most sense to me to implement it alongside the below feature request, that way the backup copy jobs can also be modified appropriately.
veeam-cloud-providers-forum-f34/configu ... 91087.html

I didn't see any other FR on the forums for this, but its been mentioned in other threads. I felt it deserves its own post. Thanks for reading and I appreciate all you do.

[chris.childerhose]

I think this is a great suggestion, and count my vote for it. Also, if you are targeting Object, it would be nice if the drop-down list for the block sizing showed the 8MB option instead of putting the registry key in to get that to show. If you are targeting block, the default options are there, and only Object unhides the 8MB option, saving you entering the Registry key, and it defaults to all backup targets now.