Slow disk digest calculations

[gollem]

Hi Guys,

I had to redo my replication jobs for our datacenter to our failover DC, now I added all VM's to a new job, started them and its going through some disk digest calculations. Since its about 10TB which I need process I wanted to use seeding as the VM's are already there.

Basic layout:
Production DC:
- Veeam Management Server (VM-A)
- 3x Veeam Proxy (VM-B,C,D), 4 concurrent tasks per proxy
- v7000 storage system over fiber to the hosts
- VM's are running on this setup

Failover DC
- 2x Veeam proxy servers (VM-E,F), 2 concurrent tasks per proxy
- v7000 storage systems over fiber to the hosts
- replica VM's are in a powered off state

Setup the job with a source proxy for VM-B,C,D and a target proxy VM-E,F
Seeding enabled, correct VM's where detected.
Data repository for replication data is set to the backup storage in the production DC
(when running normal onsite backups in the production DC i'm getting around 107MB/s when running full backups)

Jobs are running and disk digest is being calculated.
All disks (both on source and target site) in the jobs are shown as HOTADD

However its too slow, a 500GB disk took over 10 hours to calculate disk digets.

Logged on to the proxy in the failover DC, to further analyze, I see that the proxy is communicating with the host over the network (I would expected direct disk reads since its HOTADDED??). I sometimes see rates upto 800mbit/sec, but right now it doesn't even go over 80mbits/sec, so if it has to read a 2tb datadrive at 80mbit/sec that will take a long long time.

Any thoughts on where I can check what the bottleneck could be in the disk digest calculations?

Thanks

[PetrM]

Hi Erik,

I would say that there would be one of these 2 bottlenecks:
1) Data read speed from source performed by proxy is too slow
2) The link between proxy and repository with replica metadata is too slow.
If you can assign both roles to the same server: proxy and repository for metadata, it will help to exclude the version 2 (proxy-repo link) as long as digests recalculation speed remains the same.

I see that the proxy is communicating with the host over the network

Are you talking about Production or Failover data center?

But the best option is to open a support case, our support engineers will be able to review debug logs and define the bottleneck based on logs analysis. Please provide case number for our reference.

Thanks!