Exchange 2013 DAG backup is 10TB larger than 2010

[igarcia1]

Case # 01648319. Running Veeam v8.0.0.2084 and vSphere 5.5 U3. All VMDK's for database volumes are lazy-zero thick provisioned.

We recently moved from an Exchange 2010 non-DAG to Exchange 2013 DAG setup. We trimmed a lot of fat and during the migration reduced the number of servers, databases (not accounting for copies), and mailboxes. We actually have less data on the 2013 system than we did on the 2010 system, but somehow the 2013 full backup alone is larger than the entire month of backups for 2010. Some numbers:

The VBK of the 2010 backup is 10.1 TB on disk (16.6 TB data size). Daily incrementals ranged from 100 to 300 GB. We were able to store a month of backups for just over 19 TB.
The VBK of the 2013 backup is 24.7 TB on disk (50.2 TB data size). Daily incrementals so far have also been far larger than expected. At first I thought log files since we ran into some space issues and did some reseeds during the migration, so set the retention policy to 1 and took a 700 GB incremental and let it merge but the VBK size didn't change. Ready to accept a 24 TB vbk, I kicked off the next incremental on Monday just 12 hours after the last one and it came out to 300 GB. The next incremental came out to 888 GB on the following Tuesday.

Thing is, we're only storing about 6 TB of data on the 2013 mailbox databases. Including OS, DB, and Log file overhead puts us at 8 TB. Including the database copies puts us at 14 TB. So even if Veeam is failing to deduplicate our database copies, the full backup is still far larger than it should be. The Veeam tech suggested we reset CBT, which would make sense due to the logs and reseeds mentioned earlier, so made the changes and kicked off the next incremental 24 hours later on Wednesday. It's transferred 600 GB only 60% into the backup, so no fix there either. My current thinking is to move database copies around so I can get a full backup while targeting only four mailbox servers, but I don't want to make such a drastic change to the environment just yet without knowing what is causing the extra data transfer.

Any thoughts or suggestions on where to look?

[igarcia1]

Doesn't seem to be a problem with Veeam, which would indicate that somehow 19 GB of email per day is generating nearly 1 TB of changes on the environment that are recorded by CBT. We are keeping logs on the same volume as the mailbox for Auto-Reseed, but haven't completed setup for that yet (got side-tracked) but I'm not sure that would make any difference in a DAG vs non-DAG environment. It was mentioned that Veeam v9 might reduce the size of the backup by not backing up deleted blocks, but it seems too early to jump into v9 on a maybe.

[igarcia1]

Seems I've hit a wall with support. And while I can't blame them since Veeam is doing it's job, it's certainly frustrating. Current recommendation is to back up a single server with a copy of all databases. We already have two local copies for HA, so we either consume more space on third copies just for backup targets or move our secondary copies to a single server and sacrifice failure tolerance. And there's no guarantee that this will fix the problem, just a shot in the dark.

Is this the typical setup for those backing up Exchange 2013 DAG with Veeam?

[igarcia1]

Seems this is the same problem I've run into plenty of times before. Data increase followed by deletion expands the VMDK's and increased the size of the backup. I don't know why I thought a Full Backup would ignore deleted blocks. Glad that's in the new version at least. I figure reseeding will be quicker than SDelete/Zero Punch, so 32 reseeds it is. As for the incrementals, I'm hoping shifting around our database copies this weekend so Veeam only grabs a single copy of each database will reflect the changes. It seems like Veeam deduplicates the database copies in the full backup (VMDK's are 30.3TB total, backup is 24.7 TB on disk, 6 TB difference is roughly the size of our databases in Windows), but can't dedup the blocks changed by replication in the incrementals.

[chrisdearden]

Are you using bitlooker in v9?