Comprehensive data protection for all workloads
Post Reply
icaltun
Lurker
Posts: 2
Liked: never
Joined: Dec 29, 2009 12:56 pm
Full Name: Ioan Caltun
Contact:

Veeam 4.1 Slow backup of large VMDK files on VI3.5

Post by icaltun » Dec 29, 2009 1:16 pm

Hi all!

We are implementing Veeam Backup 4.1 for one of our clients.
They use ESX3.5 build 143128 and SANs EMC² CX3-10c.
Veeam is working in VCB mode and all jobs are configured SAN/NBD without CBT (obvious client will migrate to vSphere in 2010).
Powerpath is installed on the VCB/Veeam server. The VMs are hosted on the CXs and Veeam places data in FC to an AX4-5 so write speeds shouldn't be bad at all . The ClariiOns are configured for manual multi-pathing and LUN distribution between the SPs. SP workload peaks at around 60% max.

The Veeam Server is a 8 way server with 6GB of RAM. CPU usage is around 54% and RAM usage 30%

For a typical small VM i get stuff like:

7 of 7 files processed
Total VM size: 166,00 GB
Processed size: 166,00 GB
Processing rate: 121 MB/s
Backup mode: SAN/NBD without changed block tracking
Start time: 26/12/2009 18:27:06
End time: 26/12/2009 18:50:30
Duration: 0:23:23

But for a large VMDK VM it gets a lot more complicated.
7 of 13 files processed

Total VM size: 1,23 TB
Processed size: 252,21 GB
Processing rate: 41 MB/s
Backup mode: SAN/NBD without changed block tracking
Start time: 29/12/2009 12:22:12
Time remaining: 07:10:58

Yes the VMDKs are larger that 256GB. Let's say the biggest VMDK will scale up to 1.2 TB.
The design and implementation was not of my choice and optimizations of the VMs will be done next year when migrating to vSphere.

Still, is there any way I could improve these results? For me the most dangerous situation is an overwhelming snapshot ...

What I posted here is a lucky case too... I opened also a support case as I see a speed degradation over time.
The speed wil reach 10-15MB/s when my backup will be at 60%. I end up with a slow and painful backup and I am now wondering how the incrementals will perform.

These VMs are either file servers or lotus domino servers.

Any ideas or help will be appreciated.


PS: My client is more than interested in a scenario where Veeam will do VCB mode restores as well .
Today they backup every year per total 15TB of data and data growth is estimated to 30-50% per year.
That would be a different post maybe for a nice feature request :P

Kind regards,

Ioan

Gostev
SVP, Product Management
Posts: 24473
Liked: 3413 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam 4.1 Slow backup of large VMDK files on VI3.5

Post by Gostev » Dec 29, 2009 1:31 pm

This is interesting situation... I have two ideas:
1. A very large amount of changes on the large VM (will be able to say if this is true by looking at incremental size produced after this run).
2. Source or target storage issues, this is pretty much the only other thing which could explain such a slowdown. But may be devs will find out something researching the issue.

Probably live webex with devs will be best option here, I doubt they would be able to diagnose the issue through the logs and they will only show that everything works okay (just slow).

icaltun
Lurker
Posts: 2
Liked: never
Joined: Dec 29, 2009 12:56 pm
Full Name: Ioan Caltun
Contact:

Re: Veeam 4.1 Slow backup of large VMDK files on VI3.5

Post by icaltun » Dec 29, 2009 2:00 pm

Hi,

To answer the questions one by one.

1. Yes. There are several changes but only on some of the VMs : a snapshot on the biggest VM may increase by 1GB/hour. But it is only a peak estimate.

2. Of course we can do a webex whenever you like, just inform me in time so we can be present on the client site for the webex or at least a multiple webex between our team and your guys. A one day notice should be sufficient I believe.


PS: I believe we met at the VMworld in Cannes this year. Not absolutely sure. :)

Have a great day !

Ioan

Gostev
SVP, Product Management
Posts: 24473
Liked: 3413 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam 4.1 Slow backup of large VMDK files on VI3.5

Post by Gostev » Dec 29, 2009 2:26 pm

Please suggest the webex to your support engineer, and he or she will coordinate with the development folks so they could attend. Not sure when it would be the best date due to the holiday season though (dev. office has public holidays in the beginning of January).

If you met with someone from Veeam, that was not me for sure. :D

Post Reply

Who is online

Users browsing this forum: AdsBot [Google], Bing [Bot], Google [Bot] and 16 guests