Global Deduplication (global cache) Not Working

[mbruce@nirico.com]

Hello,

We have a many to one configuration at a client with numerous remote sites being backed up locally using the Windows Agent. They are backing up to a local repository that is provisioned by our VBR server at their main collocation. All sites have their own WAN Accelerator and the collocation VBR server has one. We have a backup copy job for each site, copying the windows agent backups using the site WAN accelerator as the source and the collocation VBR WAN Accelerator as the target. All backup copy jobs are backing up to the same repository in the collocation.

The backup copy jobs dedup using similar blocks in the destination repository just fine so there is some deduplication being done. However, every backup copy job always shows 0.0 B pulled from global cache. When the target WAN accelerator in the collocation is manually populated using the target repository it shows all the various OS types in the target repository and populates successfully. Yet no amount of manual population gets the global cache working properly.

I have seen jobs run in an environment where global cache does work and two events are listed during the copy job that are not present with this client:

1) Saving Digest_GUID
and
2) Global Cache Synchronization Completed

Those two events do not occur with these backup copy jobs. Both source and target WAN accelerators show as being used.

Are there particular repository/job settings that are required for global cache to work properly? I am at a loss as to why its not working.

We do have a support case open but it has been a week and no solution as of yet. Just tossing this one out there to see if anyone has seen this behavior before. I have 2 weeks to copy 20TB of remote site data into their collocation but much of it is duplicated across various sites, but modified by local staff so we have to copy it all. Most of the sites are on 5Mbps links so a functioning global cache would certainly help.

Thanks!

[foggy]

Hi Matt, what is the guest OS of the machines processed by these backup copy jobs? There are currently issues with detecting the OS type for Windows 10 and 2016 in Veeam Agent for Windows backups, which should be addressed in Veeam B&R v9.5 Update 3.

[mbruce@nirico.com]

Hi Foggy,

The OS versions being backed up are Server 2008, 2012 and 2012R2. The remote servers are all standalone Hyper-V Hosts hosting Server 2008, 2012 and 2012R2 guests. The files currently being backed up are the guest VHD\VHDX files.

Thanks,

Matt

[foggy]

Not sure I fully understand the scenario, since initially you've mentioned Veeam Agent for Windows - are you backing up Hyper-V VM's using it? Anyway, I recommend continue working with your support engineer on this and ask for the case escalation, if required. What is the case ID, btw?

[mbruce@nirico.com]

Yes, we are using the veeam windows agent to back up hyper-v guests then running backup copy jobs to get them offsite.

Case number is 02216640. Tier 2 is reviewing logs at the moment.

Thanks!

[foggy]

Aren't you using file-level backup of the entire Hyper-V host?

[Gostev]

Please note that the only supported ways to backup Hyper-V VMs are to either use Veeam Backup & Replication for Hyper-V, or have Veeam Agent installed in each protected Hyper-V VM.

[mbruce@nirico.com]

I was actually able to replicate this issue in a test environment. Spun up 3 brand new 2012R2 servers, loaded VBR on one, windows agent on the other two. Setup repositories, WAN Accelerators and backups. Just using defaults for each. Next, next, next.

Selected some local folders on the Windows Agent servers, created the backup, ran the backup copy job and 0.0 B pulled from global cache. Populated cache and still the same. 0.0 B from global cache. edit *c:\windows was one of the folders and when populating the cache the population job showed the OS as found and populated successfully*

The I added the system state to the agent backup jobs. Next copy jobs pulled 9GB from global cache and worked as expected.

It seems the system state needs to be included in windows agent jobs for global cache to work when running a backup job for them. Odd.

[mbruce@nirico.com]

Aren't you using file-level backup of the entire Hyper-V host?

Not the entire host. Folders containing the Hyper-V VMs and some shares. After my discovery that adding the system state (OS) to the Windows Agent file level backup seems to get global cache functioning when copying the agent backup over I have included the System State in the production backups. I will post results once they run.

[foggy]

The thing is global cache is used for system disks only.