Synthetic Full Backups Slow

[cffit]

I have a VM that is about 700GB in size. I realize this is large. The incremental backups for this machine go pretty fast, but the synthetic full takes about 8 hours to complete. I'm on v7 now as I saw there were others getting increased speed with v7 and synthetic full backups. Our SAN is an HP MSA P2000 and there is nothing else on here other than backup storage for VEEAM. Is this typical or is there something I can check see if I can get these synthetic fulls to go faster?

[Vitaliy S.]

Hi Christensen, what type of backup storage do you have? Is it a dedupe device by any chance?

[cffit]

No it is not. It is just a regular iSCSI storage SAN.

[foggy]

How it is added to Veeam B&R console (I mean what type of repository it is)?

[cffit]

It's an iSCSI volume that is presented as a drive letter to the local Windows 2008 R2 server. Last night it took 12 hours to create a synthetic backup of a VM that is 700GB or so and 544GB after backed up. The iSCSI SAN has nothing else connected to it other than the VEEAM server.

[foggy]

Generally, synthetic backup activity puts very high load on the target storage and time required to perform it depends mostly on the storage performance. Probably the time you see is expected for such a large full, however I would recommend to open a case with technical support so that they could review logs and verify what operations actually take most of the time during job operation.

[cffit]

Ok, I will open a case. This is an HP MSA G3 P2000 SAN with 10K SAS drives, 18 to be exact. And the only thing running to it was the synthetic full. Before v7 it was taking about half this long. Thanks

[foggy]

Keep us updated on the investigation results.

[cffit]

So support suggested I do a regular full backup since I've been doing synthetic fulls for a long time. They said after I do a "real" full backup, then I can go back to synthetics and they should run faster again. Has anyone ever heard of this?

[veremin]

Yep, “forever synthetic” scenario might result in decreased speed performance, thus, it’s recommended to run active full backup on monthly basis.

Thanks.

[cffit]

Ok, that is great to know. Is this documented anywhere or how would I ever know this?

[veremin]

From the forums, as you’ve just done. Also, this post might be helpful if you need additional clarification. Thanks.

[cffit]

I am thankful for the VEEAM forums and find them VERY helpful. I just wish some of the great information here would be noted in official documentation or an online "Best Practices" document that could be updated regularly.

And also, like the person in the other article you linked to, I didn not notice this until after the upgrade to v7, which is very odd.

[veremin]

Actually, community forums thoroughly monitored and analyzed by our technical writers team. And significant part of useful tricks is eventually written either to user guide or to best practice guide. Though, in case of newly-introduced version it might take some time, until those best practices are formed.

Anyway, thanks for the feedback; much appreciated.

Thanks.

[rrnworks]

We have a new backup job and our very first synthetic full is taking a very long time (over twice as long as the normal full backup).

[Gostev]

We have a new backup job and our very first synthetic full is taking a very long time (over twice as long as the normal full backup).

This means your backup storage is too low on IOPS capacity. Normal full backup is sequential write, even single spindle of slowest eco hard drive can handle that easily. Synthetic full, on the other hand, is lots of random IOPS. You need to chose where you want to put the load from full backup to - either you put it on your production environment by doing active (real) full backups, or you isolate it to backup storage (but this requires a storage with decent IOPS).

[BHB3805]

Is there target disk savings using a synthetic full vs. a "real" full backup?

[foggy]

No, synthetic full backup is completely identical to an active full, except the way it is created (how VM data is retrieved).

[amatteo78]

Hello,

from few days I start test Veeam B&R, I have about 15VM with 5TB disks. First backup spend about 2 hours, and consuming about 150GB on my remote storage to do it. This night Incremental spent very few time about 5 minutes each vm, but when do "synthetic full backup" spent more 4 hours. It's very slow instead first full backup.
Can I check somethings ? Thanks

M.

[Vitaliy S.]

Hi Matteo,

What do you see as bottleneck stats and what is your target storage?

Thanks!

[amatteo78]

Hello,

I have 2 NAS the first one where I have discs of my VMs, it's FreeNAS with iSCSI connect to host ESXi, it's no problem I see on stats and all fine. Then I have other NAS to put backup, this is shared NAS that I rent from my service provider where I have housing. They give me partition and I can use CIFS/NFS to share it. I don't know if it's has saturation because I can't see its stats.
But when I do first backup (full) it spend about 2 hours to do. Then when do incremental it spend few minutes every vm, but about 4 hours to do synthetic process.
Thanks

M.

[Vitaliy S.]

What is the connection link to this NAS box? If it is over the WAN, then do you have "proxying" server configured on the other side to keep traffic locally during synthetic full creation?

[amatteo78]

Yes, I connect it by WAN. what you mean with "take traffic locally" ??

[Vitaliy S.]

When you're using CIFS repository and use synthetic fulls, then traffic has to travel back and forth to create a VBK file (new full). If your WAN link is slow, then creating new VBK file can take some time.

Please see this link to know more about the option I'm referring to > http://helpcenter.veeam.com/backup/70/v ... b=proxying

[amatteo78]

Yes, I setup it. I use only 1 server to B&R server and proxy.... so I setup "this server".

[Vitaliy S.]

You should setup any Windows server as a "proxying" server on the site where your CIFS share is located, not on the source site. I don't think that you have Veeam B&R installed on the remote site, do you?

If you cannot do this, then I would suggest switching to active full backups instead. Thanks!

[amatteo78]

I have only 1 server with installed B&R and Proxy, and CIFS shred only in this proxy. So I think it's correct. but not solve problem.
Now I setup "active full backup" and check if it goes.

[Vitaliy S.]

Another way to address that is to mount this NAS box to any Windows server and then use it as a repository. Keep us updated on your testings.

[amatteo78]

sure, this night i test with "full", tomorrow I create new server windows as repository and other server as proxy. What cpu/ram need have this server, i mean proxy and repository ?
Thanks very much

M.

[foggy]

It is still not clear where this Veeam B&R server is located and what are the bottleneck stats for your backup jobs. Bottleneck stats are available by right-clicking the job and selecting the Statistic command on the shortcut menu.

System requirements for all Veeam B&R components are available in the release notes document.