Skip Calculating Digests and Just Redo thee Replication Job?

VMware specific discussions

Skip Calculating Digests and Just Redo thee Replication Job?

Veeam Logoby RPTan » Sat Jul 18, 2015 1:58 am

We noticed that whenever Veeam encounters an error with their replication job it automatically does digest recalculation. Is there a way to skip this an just redo the replication job from the last good replica? Usually it takes longer to recalculate digests, especially on big VMs (almost a day to recalculate vs. replication job of couple of hours).
RPTan
Influencer
 
Posts: 11
Liked: never
Joined: Thu Aug 18, 2011 2:07 am
Full Name: Ryan Tan

Re: Skip Calculating Digests and Just Redo thee Replication

Veeam Logoby Vitaliy S. » Sun Jul 19, 2015 8:19 pm

Hi Ryan,

If you want to redo (run active full), then you can force it via Veeam backup console by right clicking on the VM. Digest calculation is required to do an incremental pass and detect what data is missing in the target location.

On a side note, can you please tell me what's your destination site? What connection do you have to the target host?

Thanks!
Vitaliy S.
Veeam Software
 
Posts: 19945
Liked: 1144 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Skip Calculating Digests and Just Redo thee Replication

Veeam Logoby RPTan » Mon Jul 20, 2015 8:16 am

Hi Vitaly,

As we are talking about replication, what I meant is that can we redo the job from the last good replication point before the error occurred, and not the full incremental again?
Destination site at the moment is just hosts on an isolated network connected via gigabit, as this is testing before moving from Production to DR site.

On a side note, what causes slow performance for digest calculation, especially for large VMs (multi-disks of mutli-TBs each)? What performance can we expect anyway?
We use hot-add mode, with a source and target proxy, and have placed the metadata repository at the production side.

Thanks,
Ryan
RPTan
Influencer
 
Posts: 11
Liked: never
Joined: Thu Aug 18, 2011 2:07 am
Full Name: Ryan Tan

Re: Skip Calculating Digests and Just Redo thee Replication

Veeam Logoby Vitaliy S. » Mon Jul 20, 2015 1:45 pm

RPTan wrote:what I meant is that can we redo the job from the last good replication point before the error occurred, and not the full incremental again?

When digest calculation takes place, your replication job tries to do exactly this. Currently there is no way to skip this process.
RPTan wrote:On a side note, what causes slow performance for digest calculation, especially for large VMs (multi-disks of mutli-TBs each)? What performance can we expect anyway?
We use hot-add mode, with a source and target proxy, and have placed the metadata repository at the production side.

Hmm...that's a good question. I would suggest to take a look at digest calculation log file to see if there is anything suspicious or not. Our support team would be glad to assist you with task.
Vitaliy S.
Veeam Software
 
Posts: 19945
Liked: 1144 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Skip Calculating Digests and Just Redo thee Replication

Veeam Logoby foggy » Mon Jul 20, 2015 2:02 pm

Digests calculation speed directly depends on the size of the virtual disks and can take considerable time for larger VMs.
foggy
Veeam Software
 
Posts: 15275
Liked: 1131 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Skip Calculating Digests and Just Redo thee Replication

Veeam Logoby RPTan » Fri Jul 24, 2015 3:09 am

Is it safe to say that there is something definitely wrong when doing a VMWare clone (from one datastore to another) is faster than replicating via replica mapping for a large VM? or is this still expected? I'm assuming digest calculation does mainly a read on the storage and should definitely be faster right?
RPTan
Influencer
 
Posts: 11
Liked: never
Joined: Thu Aug 18, 2011 2:07 am
Full Name: Ryan Tan

Re: Skip Calculating Digests and Just Redo thee Replication

Veeam Logoby Vitaliy S. » Sun Jul 26, 2015 8:36 pm

It's almost correct, but digest calculation and then transferring data might take more time (in some cases) compared to just running active full replication job run.
Vitaliy S.
Veeam Software
 
Posts: 19945
Liked: 1144 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

[MERGED] calculating digests slower than starting from scrat

Veeam Logoby mcz » Tue Mar 14, 2017 9:05 am

Hi!

From time to time I have to increase the disk size of a particular vm which is part of a replication job. Of course, changing the disk size will cause veeam to merge all restore points (=snapshots) of the replica and to calculate the digests for just transfering the deltas between source and target vm. The thing is that this digests calculation is taking quite a lot of time and this will block other jobs (because veeam locks this machine and some proxies until replication is finished) and cpu usage is much higher.

My production and dr-site is within the same network and connected via a 1 GBit Ethernet. Therefore, creating a replica from scratch wouldn't take that much time than calculating the digests and just transfer the deltas to the target. I know that this wouldn't be the case if my dr-site wouldn't have this fast connection in between(e.g. over a wan connection), calculating the digests would then be the faster way.

So the questions are:

    why does veeam calulate the digests if production and dr-site have a fast connection to each other?
    is there a way to suppress the calculation of digests for specific jobs and replication the whole disks instead?
    are there other pro's for calculating digests over other actions?

Thank you for your help!
mcz
Enthusiast
 
Posts: 96
Liked: 9 times
Joined: Tue Jul 19, 2016 8:39 am
Full Name: Michael

Re: calculating digests slower than starting from scratch

Veeam Logoby DGrinev » Tue Mar 14, 2017 11:08 am

Hi Michael,

why does veeam calulate the digests if production and dr-site have a fast connection to each other?

Digest calculation speed depends on the size of the virtual disks. After the process changed data blocks should be transferred to the replica VM, so the traffic between sites is minimized.

is there a way to suppress the calculation of digests for specific jobs and replication the whole disks instead?

You can manually run Active Full through UI by right clicking on the VM, this could be much faster than digest calculation.

Please review this thread above for additional information. Thanks!
DGrinev
Veeam Software
 
Posts: 612
Liked: 71 times
Joined: Thu Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev

Re: calculating digests slower than starting from scratch

Veeam Logoby mcz » Tue Mar 14, 2017 12:28 pm

DGrinev wrote:You can manually run Active Full through UI by right clicking on the VM, this could be much faster than digest calculation.


Thank you for your answer, Dmitry, but I am not able to find "active full" if I right click on the vm. The Replication job (it's a job for a containter, not a single vm) doesn't show the option in the context menu and under "virtual machines" I just got the options "veeamzip", "add to replication job", "quick backup" and so on. Where should I find this option?

Thank you in advance!
mcz
Enthusiast
 
Posts: 96
Liked: 9 times
Joined: Tue Jul 19, 2016 8:39 am
Full Name: Michael

Re: Skip Calculating Digests and Just Redo thee Replication

Veeam Logoby DGrinev » Tue Mar 14, 2017 3:23 pm

Sorry if it was not clear, but in order to run active full, you should delete the target replica VM.
That's why it's not recommended way due to potential risk of losing replicas.
DGrinev
Veeam Software
 
Posts: 612
Liked: 71 times
Joined: Thu Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev

Re: Skip Calculating Digests and Just Redo thee Replication

Veeam Logoby mcz » Tue Mar 14, 2017 4:01 pm

DGrinev wrote: in order to run active full, you should delete the target replica VM.
That's why it's not recommended way due to potential risk of losing replicas.


OK, so there is currently no other way to run active full. This would be a nice feature for future updates... Running active full for a specific vm (or maybe the whole job) would be something that won't take too much time for the veeam engineers to implement :roll: :D
mcz
Enthusiast
 
Posts: 96
Liked: 9 times
Joined: Tue Jul 19, 2016 8:39 am
Full Name: Michael

Re: Skip Calculating Digests and Just Redo thee Replication

Veeam Logoby Delo123 » Tue Mar 14, 2017 4:04 pm 1 person likes this post

Maybe, but the risk is quite big. With "run active full" one would possibly not be aware that the replica on the target side would be completely lost, in case of a really big VM i'm pretty sure some people would quickly start looking for a "undo" button....
Delo123
Expert
 
Posts: 355
Liked: 102 times
Joined: Fri Dec 28, 2012 5:20 pm
Full Name: Guido Meijers

Re: Skip Calculating Digests and Just Redo thee Replication

Veeam Logoby zmedgyesi » Mon Aug 28, 2017 6:21 am 1 person likes this post

I agree with mcz. In my case, I have backup copies on the secondary site, and replicas are created from backup copies. For example, our file server's replication failed this weekend, because of a planned electric maintenance. Veeam started to calculate digests for a 1.1 TB disk, which lasted for more than 8 hours. We have a proxy on the secondary site, and my guess is, reading backup copies from its locally attached disks and writing them through FC to the storage must be much faster than calculating digests endlessly. And, in the meantime, it seems we have a few "Veeam working" snapshots under VMware - I don't know yet, how to treat them, I'm afraid they are stuck, and at the end of the day, I'll have to manually delete the VM and restart its replication. So, the end result is the same: delete affected VMs and re-replicate, and during the process, we don't have a working replica - not for a few hours but for days! Why not have a checkbox which does the same for me?
zmedgyesi
Novice
 
Posts: 5
Liked: 3 times
Joined: Sat Dec 06, 2014 10:04 pm

Re: Skip Calculating Digests and Just Redo thee Replication

Veeam Logoby antipolis » Fri Sep 01, 2017 8:12 am 1 person likes this post

+1 on this

after resizing disks on a large VM I just delete the target replica cause it's way faster to start from scratch in our environment, in this context having the extra checkbox in the rep job would save the extra step of manually deleting replica
antipolis
Enthusiast
 
Posts: 60
Liked: 7 times
Joined: Wed Oct 26, 2016 9:17 am


Return to VMware vSphere



Who is online

Users browsing this forum: BenB1978 and 1 guest