S3-integrated Storage - Block Size

[tyler.jurgens]

With S3-integrated storage (Eg: minio) the storage tells Veeam what block size to use (in this case, Minio recommends a 4 MB block).

However, when I check the job settings on a new backup job, using the defaults, the block size is set to 1 MB.

Which block size will be used on the backup job? The size the Veeam job reflects, or the size the storage system recommends?

Assuming the block size used is the block size the storage system recommends and the settings in the veeam job are ignored: If I wanted to override the block size, how is that done?

[sfirmes]

@tjurgens-s2d The block size that you specify in the backup job is what is used when putting objects to object storage. The object storage vendor may have their best practice size recommendations and if you choose to use them that is where you would select the block size to be used.

[tyler.jurgens]

On Luca's blog post (https://www.virtualtothecore.com/a-firs ... am-sosapi/) it looks like it might be planned capability to force a default block size. I'm guessing that's not in place yet?

[Gostev]

No, this has not been implemented yet. Also, some people at Veeam argue if it's even a good idea to let storage vendors control this

For example, changing the default 1MB block to 4MB will approximately double your disk space consumption in the long run. This is obviously great for a storage vendor as they get to sell 2x more nodes, but not so great for your wallet or data center footprint.

[tyler.jurgens]

As with anything, let it be configurable. If the storage vendor's SOSAPI implementation forces 4 MB blocks, let the Veeam client override it. Eg: If I create a job against a S3-integrated bucket, the block size defaults to the SOSAPI block size. If I change it, it uses the block size I set.

For our Minio implementation, we'd take 4 MB blocks over hundreds of millions of tiny blocks negatively impacting our performance. We have one client backing up 45 TB using 223 million objects! It has made my leadership team want to avoid Veeam, while I've been trying my best

The storage size issue only really gets amplified if the clients retain long term incremental backups. Yes, it takes more space, but its a tradeoff we (at least in our specific situation) need to make. Given each GFS point already is effectively free, its not so bad.

[sfirmes]

Also when the SOSAPI starts to use the object size as noted in the blog, it will be at the bucket level and every job pointed to that bucket will use that object size. As @Gostev mentioned, you can see significant storage growth when increasing the object size from the 1MB default/best practice value. Due to that, you should try sticking to changing the object size on a workload/job level and monitor the effect of the change.

[tyler.jurgens]

That's all well and good, and we've been changing block sizes on individual backup jobs. However, when you're talking many customers, each with many jobs, changing these all takes a significant amount of time.
I agree that we need to change and monitor, which we have. It just doesn't scale well and is prone to human error.

Also concerned with how Capacity/Archive tier block sizes get set. For a direct to S3 backup job, its straight forward - change the block size on the job, run an active full and call it a day. Backup Copy jobs or Capacity/Archive tier jobs are more difficult - one has to change the primary backup job block size setting.

Unless the SOSAPI knows enough to change the block size on any jobs that either have a backup copy or are part of a SOBR with offloading to S3, forcing the block size on backup jobs alone isn't going to have a major impact, as I suspect most customers will use S3 as a backup copy target, or SOBR offload target. I may be wrong here though, Veeam may have more clear metrics on how S3 is being used.