Comprehensive data protection for all workloads
Post Reply
evansdp
Novice
Posts: 4
Liked: never
Joined: Jan 14, 2016 11:37 pm
Full Name: Dan Evans
Contact:

Backup Copy slow after V9 upgrade

Post by evansdp »

There's a thread about regular backup slowness in V9 (veeam-backup-replication-f2/veeam-v9-ba ... 35255.html) and we'll be testing some of the approaches described there. But our situation is slightly different and I'm curious if anyone else is seeing this:

Our Backup Copy jobs (one in particular) have slowed to a crawl (80 hours in one case) since the V9 upgrade. Our regular Backup jobs are doing fine.

We have a single Veeam Server that also serves as a proxy plus a separate proxy server. Both Veeam servers are 2012.R2. We have two repositories on a Linux-based array with 10Gb links between the Veeam servers and repositories. All of the Backup Copy jobs point to the proxy as the bottleneck but none of the regular backup jobs do.

We're preparing to add another proxy (or 2).

Anyone else seeing this behavior?

Dan Evans
Pacific Lutheran University
Dan Evans
Pacific Lutheran University
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup Copy slow after V9 upgrade

Post by foggy »

Dan, how repositories involved in the backup copy job are added to Veeam B&R?
evansdp
Novice
Posts: 4
Liked: never
Joined: Jan 14, 2016 11:37 pm
Full Name: Dan Evans
Contact:

Re: Backup Copy slow after V9 upgrade

Post by evansdp »

Foggy-- I'm not sure of your exact question. There are 2 repositories and both were added through the Backup Infrastructure tools on the Veeam server. Only one of the repositories is involved in the backup copy jobs (we use one repository for daily, "regular" backups and the other for backup copy, "archive" jobs).
Dan Evans
Pacific Lutheran University
tsightler
VP, Product Management
Posts: 6009
Liked: 2842 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Backup Copy slow after V9 upgrade

Post by tsightler »

Any chance this is a fairly large job with a lot of VMs/disks? If so, you might want to open a case as there are a few known regressions in BCJ performance in v9 including one with an existing hotfix.

Also, it's important to note that proxies aren't really used for Backup Copy Jobs so the bottleneck stats can be a little bit misleading. Backup copy jobs are performed "repo-to-repo" so the "proxy" referred to in a BCJ is actually the source repostory.
alanbolte
Veteran
Posts: 635
Liked: 174 times
Joined: Jun 18, 2012 8:58 pm
Full Name: Alan Bolte
Contact:

Re: Backup Copy slow after V9 upgrade

Post by alanbolte »

Also make sure you don't have parallel processing disabled in the global settings.
evansdp
Novice
Posts: 4
Liked: never
Joined: Jan 14, 2016 11:37 pm
Full Name: Dan Evans
Contact:

Re: Backup Copy slow after V9 upgrade

Post by evansdp »

Thanks for the feedback so far.

@tsightler-- There's a mix of jobs-- some have only 1 or 2 VMs while others have 5-9. Sizes of VMs range from 80GB to 4TB but the amount of data processed by Veeam ranges from 2GB to 126GB. So yes, there are some "fairly large" jobs with many VMs or disks and I will be putting in a ticket. Thanks for the comment about the BCJ proxy being the source repository. That suggests we might benefit by moving our BCJ timing from ~midnight (when many of our regular backup jobs are running) to mid-day (when we have very few regular jobs running).

@alanbolte-- I checked and parallel processing is enabled.

> Dan
Dan Evans
Pacific Lutheran University
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup Copy slow after V9 upgrade

Post by foggy »

I was after the repository type used while adding the storage in Veeam B&R - Linux or CIFS? This defines where the source and target data movers are running and might explain why the proxy (effectively the source repository data mover) is reported as bottleneck.
evansdp
Novice
Posts: 4
Liked: never
Joined: Jan 14, 2016 11:37 pm
Full Name: Dan Evans
Contact:

Re: Backup Copy slow after V9 upgrade

Post by evansdp »

Both repositories are Linux (CentOS 7) and sit on the same physical box (32GB RAM with 45x3TB drive array configured with mdadm as 2 RAID6 LVMs). One volume is mounted as /raid6a and used for daily backups. Second volume is mounted as /raid6b and used to store the BCJs. Source servers are mix of Linux and Windows.

> Dan
Dan Evans
Pacific Lutheran University
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup Copy slow after V9 upgrade

Post by foggy »

As Tom has already mentioned, there were some issues with backup copy job performance after the upgrade (specifically, when target cache stopped to be used), so I recommend to contact support for verification whether it applies to your situation. Also, in your case both source and target data movers are running on the same server, this might be the reason of showing this server as the bottleneck.
Post Reply

Who is online

Users browsing this forum: ante_704, Google [Bot], morgel, restore-helper and 175 guests