Seemingly poor compression with Veeam vs. file level backup

[clipnz]

Hi, am having some trouble when moving from Backup Exec to Veeam - I seem to get getting poorer compression through Veeam than i was through backup exec, which i thought would have been the opposite?

Bit of background on the environment/etc

Have a 2node ESXi 5 cluster, running 5 virtual machines and the vcenter VM appliance
4 of the vm's are standard, single disk, windows servers
the 5th one (lets call it DC01 for the purpose of this) has the OS installed on C: (approx 100gb), then has 2 other virtual disks allocated to it, 1 of 2TB (D: Data) and 1 of 1TB (E: Archive Data). The 2tb drive is for standard data and is where the users save all their files to etc. The 1TB volume has similar data, but older/archive and is not required to be backed up by Veeam daily.

All the server's VM files are stored on data stores on an EMC VNXe 3100 SAN
This runs through a management network of 2x cisco 1gb switches, and LAN is through a standard 1gb switch

Previously, i had been backing up the 5 servers OS's through Veeam, which when doing a full backup would result in an 82gb approx file.
Backup Exec installed on the same PC as Veeam with tape drive physically in this same machine, would then backup the Veeam files + the DC01 D: drive to tape. This would all fit on disk fine, as well as including another 100gb approx server

As we want to save costs on BE remote agents, i removed the DC01 D: data from the tape job, and changed the Veeam job to backup full server of DC01, then excluded the E: drive

I then ran a full backup, and the total .vbk file size is over 1TB - so by including 600gb of data in the Veeam job, it's size has increased by 900gb+ ? Very odd

In terms of the compression/target settings in the Veeam job, compression is set to optimal and target is optimized for local storage (it is backing up to 2x 2tb disks in RAID1, on a raid controller with BBU. LSI raid card, not onboard/windows SW raid)

I have 2 questions, firstly will BE compress the .vbk file any more when it goes to tape? I was thinking it will not (the job will run tonight so i will have an answer for that).
Secondly, if i optimize the target for Network as opposed to local, and set compression to highest not optimal, is it likely i will get a much better compression ratio? Do you think the .vbk file for full backup of all servers will be around 700gb or less?

It's very strange, it seems something somewhere is blowing out the backup. There were 15gb approx of shadowcopy files on DC01 D: which i've removed, and also 50gb in recycle bin that i've removed but it seems like something else somewhere is causing a blowout - and it's odd that backup exec does not experience the same symptoms?

any help appreciated,
thanks

[J1mbo]

I'm not sure how BE was being used, but Veeam is backing up vDisk images whereas BE may have been backing up files, so once-used file-system free space will take up space with Veeam where it didn't with BE. If that was a contributing factor then it could be resolved by cleaning free space (using dd on linux or sdelete on Windows). However if the VMs are thin provisioned that WILL expand them to their maximum size at both VMFS and storage levels.

[foggy]

Nick, I would also assume that the actual size of your 2TB D: drive is much more than 600GB as you are stating. Remember that deleting the data from NTFS volume does not change the actual content of disk blocks, but only updates file allocation table records marking those blocks as "free". Since Veeam B&R does image-level backup, it treats those blocks as occupied putting them in the VBK file. To zero unused blocks so that Veeam B&R can drop those from processing, you can use sdelete utility and then run active full backup to recreate the VBK file for these changes to take effect.

[clipnz]

that sounds like the exact cause - thanks foggy/jimbo, will run sdelete before the next full backup this weekend