v11 Scale-Out with S3 - file size limits

[lahey]

Hi,

How does scaling out to S3 work on repositories in regards to file sizes?

Case:
- S3 has a size limit of 10TB pr file (at least in private cloud that we have acquired)
- Backup job full is 14TB
- Another backup job full is 22TB
- Both jobs do monthly full backups and store them for 1 year as independent restore points

Question:
- Is it possible to to scale-out these jobs with S3 archiving?
- Will the scale-out repository split the file into smaller files before hitting 10TB? (or how does this work)

[HannesK]

Hello,
it just works

Veeam files are on average size around 0.5 MB... I also recommend to check out the sticky FAQ topic and the User Guide

Best regards,
Hannes

[jim.lowry]

One thing to keep in mind, assuming I'm following your question correctly, that objects are different than files - the size limitations work differently. S3 and BLOB will both have size limits for a single object. But that doesn't mean a file has to be part of only 1 object. If you have a file larger than 10TBs, and the object protocol says an object can be only 5TB, then your 10TB file will be split into at least 2 objects. The links below will show how that works specifically with S3. All other object vendors/API's will have different values based on their specific design, but the concepts are essentially the same.

AWS S3 Multipart object limits: https://docs.aws.amazon.com/AmazonS3/la ... facts.html
AWS S3 Multipart Overview: https://docs.aws.amazon.com/AmazonS3/la ... rview.html
AWS S3 FAQ: https://aws.amazon.com/s3/faqs/