Comprehensive data protection for all workloads
m.novelli
Veeam ProPartner
Posts: 346
Liked: 40 times
Joined: Dec 29, 2009 12:48 pm
Full Name: Marco Novelli
Location: Asti - Italy
Contact:

Slow VM backup with SQL Server 2008 R2

Post by m.novelli » Jun 15, 2010 7:44 am

Hi friends, I've installed a new Vmware infrastructure with two Dell R610 servers connected to a Dell MD3000i storage (iSCSI)

I'm running vSphere 4.0 U1 and Veeam 4.1.1 with CBT

All VM are backupping at high speed (from 100 to 200 MB/sec after initial full backup) except one: a Windows 2008 R2 + SQL Server 2008 R2
This VM is backupping at 30/40 MB/sec and need from 2 to 3 hours to complete an incremental backup. During Veeam backup the SQL Server is idle. This machine produce every day about 60 GB of backups on local disks, but it's a normal quantity.

Anyone experienced something similar?

Maybe Veeam need to update the VSS integration for SQL Server 2008 R2 as they did for Exchange 2010?

Thanks for any help,

Marco

Gostev
SVP, Product Management
Posts: 23832
Liked: 3204 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Slow VM backup with SQL Server 2008 R2

Post by Gostev » Jun 15, 2010 10:20 am

Hi Marco, this could be normal for highly transactional Exchange and SQL servers to have slower incremental backup speed, because there are much more incremental data to process. This is caused by the fact that these applications use transaction logs which change a lot of disk blocks, and all these blocks have to be processed with incremental backup. Large amounts of data obviously take longer to update synthethic full with. You can find existing threads on Exchange backups about this forum as well, so this is not some specific to you. So this is something you just have to live this, obviously VMs with large amounts of changed data will always backup slower than VMs with small amount of changes (all other VMs you have).

m.novelli
Veeam ProPartner
Posts: 346
Liked: 40 times
Joined: Dec 29, 2009 12:48 pm
Full Name: Marco Novelli
Location: Asti - Italy
Contact:

Re: Slow VM backup with SQL Server 2008 R2

Post by m.novelli » Jun 26, 2010 9:18 am

I've finally solved this issue. It was a configuration issue on the storage Dell MD3000i

I have only one Disk Group (7 SAS disks x 450 GB 15k) divided in two LUN

One LUN is dedicated to SQL Server VM and one LUN is for all other VM

One LUN was owned by one RAID Controller Module (aka SP) and one LUN was owned by the other RAID Controller Module

Setting both LUN owned by the same RAID Controlled Module give me lighthing speed also on the SQL Server VM

So: for Dell MD3000i the LUNs inside a Disk Group should be owned by the same RAID Controller Module

Marco

Vitaliy S.
Product Manager
Posts: 22127
Liked: 1380 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Slow VM backup with SQL Server 2008 R2

Post by Vitaliy S. » Jun 26, 2010 9:20 am

Marco, thank you for sharing! This information is very useful.

Tom Bale
Novice
Posts: 8
Liked: never
Joined: Mar 08, 2011 10:11 am
Full Name: Thomas Bale
Contact:

Slow Exchange Backups, is this normal?

Post by Tom Bale » May 10, 2011 9:04 am

[merged into existing discussion]

Hi All,

We have now got to a point where our Veeam backup system is running without much interaction which is great but backups for our Exchange server seem to take much longer than expected. This isn't really a support issue because I want to get a feel for how others are finding this and how they have it setup.

I wouldn't say that this is a particularly heavily used exchange server (Windows 2008 R2). but backups are taking roughly 12 hours daily which means that backups are sometimes over-running into our normal opening hours.

We are using reverse incremental with optimal compression and change block tracking, is this a normal setup for an exchange server? Maybe we should not be using change block tracking?

Thanks in advance.

Vitaliy S.
Product Manager
Posts: 22127
Liked: 1380 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Slow VM backup with SQL Server 2008 R2

Post by Vitaliy S. » May 10, 2011 9:47 am

Hi Thomas,

I would say it is expected to have slower backups for VMs with lots of changed blocks during the day. By the way could you tell me what is your destination target for an Exchange backup job?

Thanks.

Tom Bale
Novice
Posts: 8
Liked: never
Joined: Mar 08, 2011 10:11 am
Full Name: Thomas Bale
Contact:

Re: Slow VM backup with SQL Server 2008 R2

Post by Tom Bale » May 10, 2011 10:30 am

Hi Vitaliy,

The backups are being stored onto local disks on the backup server. It is a physical dedicated backup server if that makes any difference. The machine is running a RAID 5 with roughly 6 disks but other back jobs are much faster and not running at the same time.

Ta

Vitaliy S.
Product Manager
Posts: 22127
Liked: 1380 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Slow VM backup with SQL Server 2008 R2

Post by Vitaliy S. » May 10, 2011 2:31 pm

Thomas, you may want to try switching to Forward Incremental backup mode, should improve the overall performance. Moreover, taking into consideration that CBT is enabled and only changed blocks are transferred, I can say that your Exchange server is fairly busy.

Could you tell me please the size of the rollbacks for this VM? By the way, you may want to check our your backup server resources/performance... What is the CPU usage during the job? If it hits the limit, you should either add more CPU resources (vCPUs) or configure this job with less compression settings, will help.

Anyway, reading back through Anton's reply above, it is expected to have slower performance rates for such VMs as Exchange/SQL Servers.

Tom Bale
Novice
Posts: 8
Liked: never
Joined: Mar 08, 2011 10:11 am
Full Name: Thomas Bale
Contact:

Re: Slow VM backup with SQL Server 2008 R2

Post by Tom Bale » May 11, 2011 3:05 pm

OK, Forward Incremental may be worth trying, would you expect to use more disk space this way?

Rollbacks are between 25GB and 40GB, is this considered large?

In terms of system resources with the Exchange jobs running it is generally sat at around 50%

Thanks

Vitaliy S.
Product Manager
Posts: 22127
Liked: 1380 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Slow VM backup with SQL Server 2008 R2

Post by Vitaliy S. » May 11, 2011 4:09 pm

Tom Bale wrote:OK, Forward Incremental may be worth trying, would you expect to use more disk space this way?
Yes, depending on your retention policy settings you will have at least two full backups (VBKs), but on the other hand the overall job performance should be much higher due to the fact that with forward incrementals you have three times less I/O, and thus much lower IOPS.
Tom Bale wrote:Rollbacks are between 25GB and 40GB, is this considered large?
No, that is not much for an Exchange server.

slidek9
Enthusiast
Posts: 44
Liked: never
Joined: Apr 20, 2010 5:00 pm
Full Name: Leon
Contact:

Re: Slow VM backup with SQL Server 2008 R2

Post by slidek9 » Jul 01, 2011 3:43 pm

Linking my thread here as we saw the same and this was my solution..
A script for backing up the latest .VBK file

stephengough
Lurker
Posts: 1
Liked: never
Joined: Jul 13, 2011 2:43 am
Full Name: Stephen Gough
Contact:

Slow Backups - With Virtual RDM

Post by stephengough » Jul 13, 2011 3:06 am

[merged]

Setup - Veeam installed on a physical machine, single quad core processor, iSCSI HBA, separate SAN switches, iSCSI SAN.
Backing up Exchange server with virtual RDM taking up to 3 days to complete.
Backup using SAN mode, CBT is enabled and the backup job does complete.
Snapshots a set to be created on separate volume with 4mb Block size
My thoughts are that it is attempting a full backup each run, but not sure how to debug further. any thoughts would be appreciated

Vitaliy S.
Product Manager
Posts: 22127
Liked: 1380 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Slow VM backup with SQL Server 2008 R2

Post by Vitaliy S. » Jul 13, 2011 8:49 am

Stephen,

Please look through this topic for more details why this behavior might be considered as expected one.

Also here is a good description of basic general troubleshooting steps that should be taken if you have a performance issue:
New user, need help optimizing backups

Thanks.

smgj
Lurker
Posts: 2
Liked: never
Joined: Jul 19, 2011 7:25 am
Full Name: Selma M. Gjeset
Contact:

Very slow performance on specific servers

Post by smgj » Jul 22, 2011 10:16 am

[merged]

I have a backup job with 7 VMs that have the same OS (2008 R2 installed on C: drive), all are exchange servers and all have a D: drive with exchange store and a E: drive with the daily store dump - store.bkf. D and E: drives are typically large drives due to large amount of mail data.

So - I have servers which are similar according to the C: drive, but that are dissimilare according to D: and E:. And the data on E: changes daily.

Which options would be best for the performance for this job (I mainly wish a faster backup, but I also do not want bloating):
- enable or disable storage deduplication? (If enabled - which compression and optimization would be best?)
- enable or disable vSphere changed block tracking?

Vitaliy S.
Product Manager
Posts: 22127
Liked: 1380 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Very slow performance on specific servers

Post by Vitaliy S. » Jul 22, 2011 10:41 am

smgj wrote:- enable or disable storage deduplication? (If enabled - which compression and optimization would be best?)
You should be using the default settings to get best performance for your jobs (deduplication enabled, compression optimal).
smgj wrote:- enable or disable vSphere changed block tracking?
In order to have smaller backup windows you definitely want to enable vSphere Changed Block Tracking.

Post Reply

Who is online

Users browsing this forum: Steve-nIP and 67 guests