"Data Domain Long Synthetic Full Creation

[asorin]

The Support Case Number is #06169051.

Hello all,

We have a backup job of our exchange server (~30tb) targeting a primary repository with refs and a backup copy job from this repository to a DataDomain for longer term storage.
Since we are using gfs on the backup copy job, a synthetic full should be created on the DataDomain each Sunday. The daily incremental to the DD is about 2 hours. However the creation of the synthetic full on DD takes more than 4 days. We are trying to understand why is so much time needed for this process and what are the involved parties. We are unable to spot any high usage of cpu/bandwith on backup server/gateway server/data domain. Like where is the bottleneck for this process?

Thank you for any advice,

Sorin.

[Andreas Neufert]

Did you ensured that the Repository Gateway Server is at the same site/network as the DataDomain?

[karsten123]

please consider a active full.
further info: kb1745

[asorin]

Did you ensured that the Repository Gateway Server is at the same site/network as the DataDomain?

Because we write to DD from different sources, we have the Repository Gateway Server for the DD configured on auto. It works in terms that for other copy jobs the expected server is selected.

We don't have a Gateway Server (or any other Veeam component) deployed in the DD network.

[Andreas Neufert]

OK please check in the job details (or maybe in the logs ... with support) what server was used. Potentially you copy things around over wrong places.
Potentially consider to place the Repository server next to the DD and use it for any source?

As it is backup copy job, I think the mount server is used for creation of the SF. Which might not be ideal placement. As well the backup server itself is fallback if mount server is not accessible.

[asorin]

please consider a active full.
further info: kb1745

We have just configured a weekly active full, however we will know how it works in about a week time.

[Andreas Neufert]

I think based on the Auto selection of the gateway server you might face more a network bottleneck issue then a SF performance issue.
Go into the Job Statistics (Or History button at the bottom of the left menu) you can navigate with the < or > keys back an forth in time and go to the last synthetic full. It will tell you what gateway server was selected. Then please check if the data flow goes ideally. Potentially you transfer back and forth the data over a WAN link twice or so.

[karsten123]

SF with dedup appliances is most of the time a bad idea and painfully slow.
write once and read never is the best strategy for this scenario, imho.
optimal network path are a base requirement, of course.

[Andreas Neufert]

Yes, I agree. Usually SF processing takes somewhere around 80-90% of the time that a Active Full processing would take. If the dedup appliances are very full or very long term operation then it can take even longer. It really depends on the situation, model and setup.

So if you write 5% change rate into the system, then the SF processing takes usually 10-20x longer.

But I did not get any comaplaint/escalation of DD being slow since a long time, this is why I think something else is affecting performance => non ideal data path is my suspision.

[Andanet]

As Karsten wrote BP for deduplication appliance Is to use Active full. SF uses at keast 8 streams for every backup to create It. And all depends from Total streams DD has. If all stream are used other SF are queued.

[splaxco]

We've been using DD with Veeam for almost 5 years. Synthetic full backups work well enough for smaller (<100 VMs) environments but anything larger and we have to switch to weekly active full backups. I have no proof but suspect it's inefficient use of the DD Boost plug-in from the proxy/gateway servers. I have not seen slow synthetic full backups from other software products with DD as a target.

[Andreas Neufert]

Other backup solutions have other data formats. Ours look more like a database with metadata and so on present, so our write/read patterns run way better on storages that are build for random IO workloads.