-
- Influencer
- Posts: 12
- Liked: 3 times
- Joined: Jun 17, 2015 2:43 am
- Full Name: David Rubic
- Contact:
Calculating digests issue
Hi Guys,
I have a Linux VM that I replicate hourly. 2 nights in a row now the replication has taken 4 hours+ to replicate locally, it gets held up for a very long time on calculating digests of the hard drive.
I am replicating locally. My B&R Server/Proxy is in my head office (10.0.5.0 VLAN) and I replicate to the DR ESXi hosts in the building across the road connected via fibre on a different subnet (10.0.6.0 VLAN)
All the other VM's I replicate hourly function fine, yet this one has just started giving me troubles and I can't afford to be waiting 3 hours at the minimum. We are using V9 with the latest patch applied.
Can anyone offer any insight?
I have a Linux VM that I replicate hourly. 2 nights in a row now the replication has taken 4 hours+ to replicate locally, it gets held up for a very long time on calculating digests of the hard drive.
I am replicating locally. My B&R Server/Proxy is in my head office (10.0.5.0 VLAN) and I replicate to the DR ESXi hosts in the building across the road connected via fibre on a different subnet (10.0.6.0 VLAN)
All the other VM's I replicate hourly function fine, yet this one has just started giving me troubles and I can't afford to be waiting 3 hours at the minimum. We are using V9 with the latest patch applied.
Can anyone offer any insight?
-
- Veeam Software
- Posts: 21138
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Calculating digests issue
David, haven't you recently re-sized any of the source VM disks?
-
- Influencer
- Posts: 12
- Liked: 3 times
- Joined: Jun 17, 2015 2:43 am
- Full Name: David Rubic
- Contact:
Re: Calculating digests issue
Hi Jim, no we haven't. I've noticed another 2 VM's having this issue also now
-
- VP, Product Management
- Posts: 27375
- Liked: 2799 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: Calculating digests issue
David, maybe you have re-attached these disks recently? The reason for digest recalculation can be any modification of the source disk, which seems to happen in your case. Let us know what the subsequent job run will be, digest recalculation should not be happening.
On a side note, If you want to track changes for the virtual infrastructure configuration, please take a look at these Veeam ONE reports > VMware Configuration Tracking Report Pack
On a side note, If you want to track changes for the virtual infrastructure configuration, please take a look at these Veeam ONE reports > VMware Configuration Tracking Report Pack
-
- Influencer
- Posts: 12
- Liked: 3 times
- Joined: Jun 17, 2015 2:43 am
- Full Name: David Rubic
- Contact:
Re: Calculating digests issue
Nope, no disk reattaching, there has been no changes in our virtual environment for at least 3-5 months, and prior to upgrading to V9 I wasn't having this issue (could be completely unrelated though, so im not blaming the upgrade)
Also, the issue is happening on multiple VM's every single night some taking 45 mins minimum to calculate, some taking up to 5 hours
I'll take a look at the link
Also, the issue is happening on multiple VM's every single night some taking 45 mins minimum to calculate, some taking up to 5 hours
I'll take a look at the link
-
- VP, Product Management
- Posts: 27375
- Liked: 2799 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: Calculating digests issue
If nothing pops up after quick investigation, then ask our support team to look through backup job logs. There should contain some relevant information as well.
-
- Expert
- Posts: 158
- Liked: 30 times
- Joined: Dec 05, 2010 9:29 am
- Full Name: Bob Eadie
- Contact:
Re: Calculating digests issue
After moving to a new VC (and running the migration tool from support) and remapping our VMs, I knew to expect a 'calculating disk digests', but these seem painfully slow, even to the extent that I am wondering whether a new full replication might be quicker.
We have two VMs which are each about 3 TB, made up of 6 x 512GB VHDs. Each of these 512GB is taking between 4 and 7 hours to calculate disk digests - which means the whole 3TB is going to take at least 24 hours . . .
Is this expected, or should I raise a support call?
We have two VMs which are each about 3 TB, made up of 6 x 512GB VHDs. Each of these 512GB is taking between 4 and 7 hours to calculate disk digests - which means the whole 3TB is going to take at least 24 hours . . .
Is this expected, or should I raise a support call?
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
-
- Veteran
- Posts: 7328
- Liked: 781 times
- Joined: May 21, 2014 11:03 am
- Full Name: Nikita Shestakov
- Location: Prague
- Contact:
Re: Calculating digests issue
Hello Bob,
The migration tool doesn`t help with replication jobs so digests need to be recalculated what indeed takes significant amount of time.
So yes, that`s an expected behavior.
The migration tool doesn`t help with replication jobs so digests need to be recalculated what indeed takes significant amount of time.
So yes, that`s an expected behavior.
-
- Veeam Software
- Posts: 21138
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Calculating digests issue
Digests calculation depends on the data size. I remember previously it took roughly 1 minute to calculate digests for 1GB of data, though this was improved in one of the latest versions. But still, considering the size of your VMs, I'd say you're seeing an expected performance.
-
- Expert
- Posts: 158
- Liked: 30 times
- Joined: Dec 05, 2010 9:29 am
- Full Name: Bob Eadie
- Contact:
Re: Calculating digests issue
OK thanks both, I will be patient. I might do an experiment to find out if starting again with a fresh full might be quicker.
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
-
- Veteran
- Posts: 7328
- Liked: 781 times
- Joined: May 21, 2014 11:03 am
- Full Name: Nikita Shestakov
- Location: Prague
- Contact:
Re: Calculating digests issue
It would be quicker if you created a new job since recalculation takes more time in most cases.
-
- Novice
- Posts: 3
- Liked: 1 time
- Joined: Aug 20, 2018 8:50 am
- Full Name: Aidan IRE
- Contact:
Re: Calculating digests issue
Hi just checking did you ever resolve this issue? I have had issues with this problem and i think you are correct it is related to an upgrade to v9. I have 2 companies pretty much same setup same number same type of vms (mix of linux and windows)
Each site has on site reps, on site backups and offsite reps.
The site with v8 rarely gives any trouble. The V9 site very frequently especially for linux is calculating disgests and so the jobs take longer, on top of this the first rep of the on site the next morning has "cannot find dirty block infomation in previous restore point, cbt will not be used", it then calulcated digests again, so a job that should take 5 to 10 min, is 2 to 3 hours. I have logged another call with veeam but while researching issue came across your post
Thks
Each site has on site reps, on site backups and offsite reps.
The site with v8 rarely gives any trouble. The V9 site very frequently especially for linux is calculating disgests and so the jobs take longer, on top of this the first rep of the on site the next morning has "cannot find dirty block infomation in previous restore point, cbt will not be used", it then calulcated digests again, so a job that should take 5 to 10 min, is 2 to 3 hours. I have logged another call with veeam but while researching issue came across your post
Thks
-
- Veeam Software
- Posts: 21138
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Calculating digests issue
Hi Aidan, we cannot check the solution to David's issue as he didn't share the case ID, so please continue investigating with your support engineer.
-
- Expert
- Posts: 206
- Liked: 41 times
- Joined: Nov 01, 2017 8:52 pm
- Full Name: blake dufour
- Contact:
Re: Calculating digests issue
what kind of disk do u have provisioned?
-
- Novice
- Posts: 3
- Liked: 1 time
- Joined: Aug 20, 2018 8:50 am
- Full Name: Aidan IRE
- Contact:
Re: Calculating digests issue
4 disks, thick provisioned lazy zeroed. There is maintenance that goes on during the night, but i disabled the off site rep, once off site disabled, the on site rep and backup run fine. Windows vms no problem with onsite/offsite/backup.
-
- Influencer
- Posts: 12
- Liked: 3 times
- Joined: Jun 17, 2015 2:43 am
- Full Name: David Rubic
- Contact:
Re: Calculating digests issue
Hi Foggy, I believe my case ID was Case # 01891991. We resolved this issue with our engineer.foggy wrote:Hi Aidan, we cannot check the solution to David's issue as he didn't share the case ID, so please continue investigating with your support engineer.
-
- Service Provider
- Posts: 248
- Liked: 28 times
- Joined: Dec 14, 2015 8:20 pm
- Full Name: Mehmet Istanbullu
- Location: Türkiye
- Contact:
Re: Calculating digests issue
Hi everyone
I'm experienced this issue before. I start replicate one VM. Incremental Replication finished successfully about 5 minutes, then i started again. Nothing changed source VM. Then calculating digest and this process take a about 1 hour.
I think if replication cycles too short Veeam or Vmware snapshot API detect changes. I don't want open the case
I'm experienced this issue before. I start replicate one VM. Incremental Replication finished successfully about 5 minutes, then i started again. Nothing changed source VM. Then calculating digest and this process take a about 1 hour.
I think if replication cycles too short Veeam or Vmware snapshot API detect changes. I don't want open the case
VMCA v12
-
- Veeam Software
- Posts: 21138
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Calculating digests issue
As far as I can see from the case notes, only completely recreating the jobs helped to stop digests calculation.dvdrbc wrote:Hi Foggy, I believe my case ID was Case # 01891991. We resolved this issue with our engineer.
-
- Novice
- Posts: 3
- Liked: 1 time
- Joined: Aug 20, 2018 8:50 am
- Full Name: Aidan IRE
- Contact:
Re: Calculating digests issue
Hi just to update on my case 03151774, when i gave the previous case number with this thread assigned engineer said
"Root cause for that case was that vmREFs were identical for 2 different target hosts. Veeam creates a temporal folder for each vm that should look like *repository*\*jobname*\*vmREF* and place replica metadata here. If vmREF is the same for 2 hosts then first metadata will get overwritten at the start of the second one."
So to validate id did below for on site and dr rep vmware host, after going to
1. https://*vcenter_name*/mob , if you're using standalone hosts then it will be ESXi host ip
2. Properites -> content
3. RootFolder
4. Datastore
5. childEntity
6. Datastore (select datastore where VM lies)
7. Check vmREF for VMs at the bottom
The vmref for the replicated vm on each host was the same (1), which caused the digest issue.
On the DR rep job, changed the medadata directory for the job. Since this change all backup/reps working perfectly.
Aidan
"Root cause for that case was that vmREFs were identical for 2 different target hosts. Veeam creates a temporal folder for each vm that should look like *repository*\*jobname*\*vmREF* and place replica metadata here. If vmREF is the same for 2 hosts then first metadata will get overwritten at the start of the second one."
So to validate id did below for on site and dr rep vmware host, after going to
1. https://*vcenter_name*/mob , if you're using standalone hosts then it will be ESXi host ip
2. Properites -> content
3. RootFolder
4. Datastore
5. childEntity
6. Datastore (select datastore where VM lies)
7. Check vmREF for VMs at the bottom
The vmref for the replicated vm on each host was the same (1), which caused the digest issue.
On the DR rep job, changed the medadata directory for the job. Since this change all backup/reps working perfectly.
Aidan
Who is online
Users browsing this forum: No registered users and 71 guests