Decreasing Backup Speed During Active Full

[bazzajok]

Hi,

After having to re-install our Veeam Server from scratch we are faced with an issue which we never had before!

During active full backups we see decreased performance through the time of the job which has taken our active full times from 16 hours to over 50 hours! Obviously this isn't good as we can't fit this into a weekend anymore.

I've seen other similair posts on here but the issues seemed to surround v6.5 and we are running v7 patch 4.

http://forums.veeam.com/vmware-vsphere- ... 04-30.html

This issue is a real pain and I'm struggling to see what the issue is. The issue seems to surround the Veeam server as it has happened during a backup to tape too.

I've linked a screenshot highlighting the issue. Any pointers would be great.

http://1drv.ms/1FNAhNp

The call has been logged with support Case # 00691513 but nothing sorted as of yet.

Thanks

Lee

[nefes]

Do you have latency control feature enabled by any chance?
It can be found in I/O Control tab in options.

[Vitaliy S.]

Based on the screenshot I can see that most of the time is consumed by VM preparation task. Did it take the same amount of time before you decided to re-install Veeam backup server?

[bazzajok]

Thanks for the replies.

We don't have the I/O control option in version 7 do we?

Thanks

Lee

[foggy]

No, it was first introduced in v8.

[bazzajok]

The long time spent preparing next VM for processing isn't something I've noticed it was more just the overall length of time.

Does this time include the time spent waiting for backup resources?

Lee

[Vitaliy S.]

No, it does not include this time. Let's see what our engineer can say after reviewing your debug log files. BTW, did you notice any performance decrease during incremental job runs?

[bazzajok]

Hi,

They are OK, typically just over 1 hour it's the fulls which are killing me. I know we have some work to do on our infrastructure to increase the speeds but if we could at least keep a consistent speed it would help!

Thanks

Lee

[Vitaliy S.]

Have you tried switching to different transport mode or creating job with less amount of VMs? Do you see the same issue? Just trying to isolate the problem.

[bazzajok]

Hi,

No, I think I'll have to try changing things slowly like you say to isolate the issue.

Thanks

Lee

[Vitaliy S.]

Please keep us posted on your findings.

[bazzajok]

Hi,

After a server reboot the full backup over the weekend processed quicker but still poor with the same symptoms of the speed dropping off over time.

I am now running the full backup to tape which normally takes around 5-6 hours and this has done the same (see link). The Veeam server seems to have a high number of Pages/Sec and this is constant.

http://1drv.ms/1yyaULy

I am starting to wonder if I should rebuild the server with Server 2012 or 2008 R2.

Thanks

Lee

[foggy]

The amount of cached memory makes me think this is very similar to that older issue. I will pass your case number to R&D for further investigation.

[tsightler]

I am starting to wonder if I should rebuild the server with Server 2012 or 2008 R2.

What version of Windows is the system currently and how much memory is in it? I'd certainly recommend 2012 or newer as old Windows versions had some serious cache issues that could cause the behavior you are seeing.

[bazzajok]

Foggy, thanks for that.

We are running Windows Server 2008 SP2 and the server has 72GB RAM (it was originally a VM host).

Thanks

Lee