How to work around slow upload speed on initial sync?

[flomp]

I have a fairly simple setup (I think)

• SOBR repository constisting of
  • Performance Tier: local disk
  • Capacity Tier: Wasabi S3
• Backup Job (VMs, and a few agent jobs) backing up to the SOBR nightly from Monday-Saturday
• Backup Copy Job, copying from the SOBR to rotated USB drives

Some numbers:

• The initial backup size was about 700GiB.
• Unfortunately, the internet connection is only 8Mbit (symmetric). It is not possible to sync to Wasabi during the day as this fills up the upstream and makes the internet connection so slow and unreliable, that work is not possible anymore.
• Therefore, it is only possible to upload about 29GiB nightly
• Also, new backup data is about 25-35 GiB daily.
• Only on Sundays, when there is no backup taken and when it is possible to upload the whole day, it is possible to upload more data than new data is created. We manage to catch up ~30-50 GiB every week.

Therefore, it will take months until everything is uploaded.

The question is: How can I speed this up?
I thoght about creating an additional "normal" local repository and setting it as destination for all backup jobs. Then, the SOBR repo will not grow anymore. Once the SOBR is fully synced to S3, I change the destination for all backup jobs back to the SOBR. Of course, the next backup will increase the size of the SOBR repo more than the usual 25-35 GiB - but hopefully not 700 GiB.
If neccessary, repeat (I hope not...)

Is this possible? Is it reasonable?

[Mildur]

Hi flomp

The initial backup size was about 700GiB.

Therefore, it is only possible to upload about 29GiB nightly
Also, new backup data is about 25-35 GiB daily.

The question is: How can I speed this up?

If you want to use a Cloud Service, you must make sure that you have the required bandwidth.
If you can't offload the daily restore points to the object storage, there is a chance that you cannot meet your RPO Requirements in case something happens to the production environment and performance tier. Just wanted to mention that.

Once the SOBR is fully synced to S3, I change the destination for all backup jobs back to the SOBR. Of course, the next backup will increase the size of the SOBR repo more than the usual 25-35 GiB - but hopefully not 700 GiB.

Theoretically, it could work. If you map the backup jobs to the old backup chains on the SOBR, it should be possible to reuse them. Do you have the chance to run a test with a new backup job and a smaller vm?

[flomp]

Thanks Mildur,
yes we know that this is not optimal - However, when the S3 storage is up-to-date eventually, we would at least manage to always catch up over the weekend, which is better than doing no cloud backup at all...

I will think about testing this with a small VM and a test-SOBR/S3-bucket.

[jorgedlcruz]

As Fabian said already, that speed will make everything really difficult to leverage an Object Storage on the Public Cloud.

Another option you have is to use the Throttling during business hours, so at least you can offload a 4Mbps during the 8/10 working hours, and full speed during the rest of the day. This will give extra GiB.

[flomp]

Hi Jorge,
thank you! I will try Throttling first!

[jorgedlcruz]

Give it a try, it might help. In case you are wondering how to add the throttling correctly, this is an example with some working hours Monday/Friday: