Comprehensive data protection for all workloads
Post Reply
Kellen.Flood
Enthusiast
Posts: 51
Liked: 3 times
Joined: Apr 19, 2013 8:30 pm
Full Name: Kellen Flood
Contact:

Backup Copy Job 60 Hour merge HELP

Post by Kellen.Flood »

Hey All,

I am currently backing up from my repository to AWS S3 via the gateway caching mechanism, but it is currently taking over 60 hours when merging the restore point (creating GFS restore point in s3.

Merging oldest restore point into full backup file (18% done) 59:16:48
Required backup infrastructure resources have been assigned
Creating Synthetic full restore point 7/27/2016 59:15:42

Any advice on why this is taking so long?

We will be opening the support case shortly
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Backup Copy Job 60 Hour merge HELP

Post by tsightler » 5 people like this post

Probably because it is having to pull all of the data back from S3 to write to the new full backup. This will be a very slow process, and likely costly as well. Since you say you are doing GFS, I assume that means you are running backup copy jobs to the S3 gateway. If so, you should probably choose the option to read entire restore point from source backup instead of trying to synthesize it. This will copy the new full from the source backup repository, which I assume is on local disk rather than than the S3. This will be much faster than trying to synthesize a backup with data that has to pull blocks back down from S3.
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Backup Copy Job 60 Hour merge HELP

Post by Gostev » 2 people like this post

Yes, what Tom recommends would help big times in this case for sure - but be prepared that either way, we are talking here about periodic fulls that would have to be uploaded to Amazon - and this may take significant time regardless. Should be fine for small backups of course, but is problematic for bigger data sets.

Efficient forever-incremental backup to the cloud was actually the reason why we introduced Veeam Cloud Connect and discontinued our "Cloud Edition", which did support Amazon as a target.
ckent
Enthusiast
Posts: 43
Liked: never
Joined: Sep 03, 2015 8:00 am
Contact:

Re: Backup Copy Job 60 Hour merge HELP

Post by ckent »

i had a very similar issue using cloud connect.
what no one is doing is offering a solution.

How does the OP get all his backup into the cloud without having to perform what is effective a full seed every week ?
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Backup Copy Job 60 Hour merge HELP

Post by Gostev »

With Cloud Connect, GFS fulls are synthesized in the cloud - tenant's components are not involved, so there's no traffic at all.
drichman
Influencer
Posts: 19
Liked: never
Joined: Jan 24, 2011 5:22 pm
Full Name: Dean Richman
Contact:

Re: Backup Copy Job 60 Hour merge HELP

Post by drichman »

Gostev wrote:With Cloud Connect, GFS fulls are synthesized in the cloud - tenant's components are not involved, so there's no traffic at all.
I am using Cloud Connect, but still have VERY long times to complete the cloud based sync. When creating a restore point in Azure, it can take 18+ hours to complete the process with an 8-10Tb source disk. I assume this is due to the size of the disk, number of files and available resources in Azure. I have been working with Veeam staff to speed things up, but we have not found a solution yet (other than shrinking the size of the backup data, which we are working on).

Let me know if anyone has an easier option.
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Backup Copy Job 60 Hour merge HELP

Post by Gostev »

Well, ultimately it comes down to backup storage performance of course, and probably your Azure VM size does not have too many IOPS included.

While we can't really accelerate your backup storage of choice, we will have one new feature in 9.5 that will make synthetic fulls work extremely fast while requiring little to no IOPS from backup storage. It's really game changing - stay tuned for more info soon!
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Backup Copy Job 60 Hour merge HELP

Post by dellock6 »

(such a teaser.... :D)
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
drichman
Influencer
Posts: 19
Liked: never
Joined: Jan 24, 2011 5:22 pm
Full Name: Dean Richman
Contact:

Re: Backup Copy Job 60 Hour merge HELP

Post by drichman »

Gostev wrote:Well, ultimately it comes down to backup storage performance of course, and probably your Azure VM size does not have too many IOPS included.

While we can't really accelerate your backup storage of choice, we will have one new feature in 9.5 that will make synthetic fulls work extremely fast while requiring little to no IOPS from backup storage. It's really game changing - stay tuned for more info soon!
We are about to ditch our Azure/CloudConnect environment for the same reason. When can we expect this revolutionary release? Also, will there be anything to address Veeam not being able to natively access Azure cold blob storage?
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Backup Copy Job 60 Hour merge HELP

Post by Gostev »

We are planning to release the new version in October.
Kellen.Flood
Enthusiast
Posts: 51
Liked: 3 times
Joined: Apr 19, 2013 8:30 pm
Full Name: Kellen Flood
Contact:

Re: Backup Copy Job 60 Hour merge HELP

Post by Kellen.Flood »

Thanks sightler. That worked for us
Post Reply

Who is online

Users browsing this forum: Amazon [Bot], ante_704, aurelien-beneteau, Bing [Bot] and 286 guests