Backup Copy Job 60 Hour merge HELP

[Kellen.Flood]

Hey All,

I am currently backing up from my repository to AWS S3 via the gateway caching mechanism, but it is currently taking over 60 hours when merging the restore point (creating GFS restore point in s3.

Merging oldest restore point into full backup file (18% done) 59:16:48
Required backup infrastructure resources have been assigned
Creating Synthetic full restore point 7/27/2016 59:15:42

Any advice on why this is taking so long?

We will be opening the support case shortly

[tsightler]

Probably because it is having to pull all of the data back from S3 to write to the new full backup. This will be a very slow process, and likely costly as well. Since you say you are doing GFS, I assume that means you are running backup copy jobs to the S3 gateway. If so, you should probably choose the option to read entire restore point from source backup instead of trying to synthesize it. This will copy the new full from the source backup repository, which I assume is on local disk rather than than the S3. This will be much faster than trying to synthesize a backup with data that has to pull blocks back down from S3.

[Gostev]

Yes, what Tom recommends would help big times in this case for sure - but be prepared that either way, we are talking here about periodic fulls that would have to be uploaded to Amazon - and this may take significant time regardless. Should be fine for small backups of course, but is problematic for bigger data sets.

Efficient forever-incremental backup to the cloud was actually the reason why we introduced Veeam Cloud Connect and discontinued our "Cloud Edition", which did support Amazon as a target.

[ckent]

i had a very similar issue using cloud connect.
what no one is doing is offering a solution.

How does the OP get all his backup into the cloud without having to perform what is effective a full seed every week ?

[Gostev]

With Cloud Connect, GFS fulls are synthesized in the cloud - tenant's components are not involved, so there's no traffic at all.

[drichman]

With Cloud Connect, GFS fulls are synthesized in the cloud - tenant's components are not involved, so there's no traffic at all.

I am using Cloud Connect, but still have VERY long times to complete the cloud based sync. When creating a restore point in Azure, it can take 18+ hours to complete the process with an 8-10Tb source disk. I assume this is due to the size of the disk, number of files and available resources in Azure. I have been working with Veeam staff to speed things up, but we have not found a solution yet (other than shrinking the size of the backup data, which we are working on).

Let me know if anyone has an easier option.

[Gostev]

Well, ultimately it comes down to backup storage performance of course, and probably your Azure VM size does not have too many IOPS included.

While we can't really accelerate your backup storage of choice, we will have one new feature in 9.5 that will make synthetic fulls work extremely fast while requiring little to no IOPS from backup storage. It's really game changing - stay tuned for more info soon!

[dellock6]

(such a teaser.... )

[drichman]

Well, ultimately it comes down to backup storage performance of course, and probably your Azure VM size does not have too many IOPS included.

While we can't really accelerate your backup storage of choice, we will have one new feature in 9.5 that will make synthetic fulls work extremely fast while requiring little to no IOPS from backup storage. It's really game changing - stay tuned for more info soon!

We are about to ditch our Azure/CloudConnect environment for the same reason. When can we expect this revolutionary release? Also, will there be anything to address Veeam not being able to natively access Azure cold blob storage?

[Gostev]

We are planning to release the new version in October.

[Kellen.Flood]

Thanks sightler. That worked for us