Veeam B&R 5 : SQL VM takes 27 hours to backup : ID#5211811

[sacd]

Hi,

My infrastruture consists of 2 ESXi hosts (ESXi 4.1.0, 721871) (IBM x3850 x5 48 logical cores, 256GB), a vSphere server (16 Core, 8GB Ram, 2.5 TB disk), all connected to fibre attached SAN.

I backup all VM's to the local storage on the Vsphere 4.1/Veeam 5 server on fridays, and replicate to a remote DRP (essentially a LAN segment, same class B range, connected to core network by 2GB Fibre) on Saturdays.

Most VM's backup quite quickly, but my development SQL VM (4 CPU, 16GB) this last run, took 27 hours to backup. The replication job replicated this server is 8 hours.

This server has 5 disks; the 5th disk (1.7TB) is excluded from the backup, and is set to independent:persistant.

This backup job has the settings below
Direct SAN access, with failover to network
Reversed Incremental
Inline data dedup
Optimal Compression
Optimized for Local Target
Use changed block tracking data and Enabled changed block tracking for all Processed VM's
Enable automatic backup integrity checks

Application aware backup processing is enabled, and set to NOT truncate logs

I got feedback from support to enable jumbo frames on the SAN network, but this is a Fibre attached SAN? Also, nothing has changed from a configuration standpoint, so I'm reluctant to make any changes to any configs for now.

If anyone else has experienced this sort of issue, please let me know.

thank you.

[foggy]

Could you please look at the bottleneck statistics for the last backup and replication jobs run, and also check whether the Direct SAN mode was effectively used during last backup job run (for that, right-click the session, select Statistics and click the VM in the list, you will see the transport mode label ([san] or [nbd]) right after the source proxy name selected for processing)?

[sacd]

Hi foggy,

Below is the text copied from that window

===================Start========================

7 of 8 files processed

Total VM size: 1.95 TB
Processed size: 1.95 TB
Processing rate: 21 MB/s
Backup mode: SAN/NBD with changed block tracking
Start time: 17/08/2012 21:43:39
End time: 19/08/2012 01:12:35
Duration: 27:28:56


Verifying changed block tracking...
Disk "Hard disk 5" has incorrect changed block tracking configuration.

Retrieving VM disks information...
Disk '[hodat009-gs5] hosqldev2008/hosqldev2008.vmdk' has been skipped because it was excluded from processing by user

Backing up object "[hodat009-gs5] hosqldev2008/hosqldev2008.vmdk"

One or more VM disks have incorrect changed block tracking configuration. To resolve this, open VMware vSphere Client, right-click the VM, choose Edit Settings, Options tab, select General, click Configuration Parameters, and set all entries with ‘ctkEnabled’ substring to false. Veeam Backup will then automatically re-enable changed block tracking with the correct settings during the next job run.

=================END====================

Bear in mind that the disk hodat009-gs5] hosqldev2008/hosqldev2008.vmdk is excluded from the backup.

Thanks for your input.

[Gostev]

This is not the information we need. Please follow the instructions above carefully to get to bottleneck stats and processing mode indicator.

[sacd]

If I go to sessions, right click the session in qustion, I don't have the Statistics option, I only see Stop session (greyed out), Details and HTML Report. When I click on details, it presents exactly the same information as before. I'm on version 5 of Veeam?

[foggy]

Ah, sorry, thought it was 6.x. Then please right-click the job, select Realtime Statistics, and select the particular VM to the left. You can check the processing mode there. No bottleneck stats for this version, but still we can reject the obvious assumption that the job failed over to network mode.

[Gostev]

You should really upgrade to the latest version. Technical support for v5 ended 2 months ago...

Unfortunately, v5 has no facility to help troubleshoot your performance issue easily.
Bottleneck detection was only added in version 6.0

[sacd]

Damn! I had upgraded to V6, but was never able to get the replication to work http://forums.veeam.com/viewtopic.php?f=24&t=9738
I'm puttng a project together to upgrade to VMware ESXi 5.1 and Veaam 6, so I may just shelve this issue then.

Back to the issue though, I don't think it failed over to network (Backup mode: SAN/NBD with changed block tracking)

Thanks very much for your feedback and time, appreciate it.

[foggy]

Back to the issue though, I don't think it failed over to network (Backup mode: SAN/NBD with changed block tracking)

So to be sure, have you got the [san] label after the source proxy name in the realtime stats window?

[Gostev]

v5 does not show this tag. However, it would have reported failover to network.