v6 Backup File Format & dedupe appliances

[dadge]

Hi, does v6 have a radically different backup file format to v5? We have a DataDomain box and since the upgrade to v6 we are using way more space than we used to. We can only assume this would be because the backup files consist of two formats - old and new - and is having to keep the same data for both?

Is this likely?

Thanks.

[foggy]

The format is new but no such radical changes. Do you mean that the VBK file for the same job is much larger than it was in v5? Have you maybe tuned some compression/deduplication settings after the upgrade? Also, you could compare the size of backups produced by the upgraded v5 job with a similar brand new v6 job to check if this makes any difference. Thanks.

[dadge]

No, I mean that the backup file sizes are more or less the same but I'm not seeing the same dedupe ration on the appliance, like the data in the new files is different to the data in the old files so it stores the data twice, e.g.

1 v5 backup 1tb + 1 identical v5 backup 1tb = 1tb stored
1 v5 backup 1tb + 1 identical v6 backup 1tb = 2tb stored.

Its obviously not as clear cut as this but we are definitely not able to store as many backups on the Data Domain now although it is currently holding backups from before and after the upgrade. Jobs themselves are unchanged.

[foggy]

Ah, gotcha. Then the format changes could really be the reason, I suppose. Two backup files of the same set of VMs having similar format will dedupe very well while files with two different data layouts will have much less identical blocks to be deduped. The required space will be reduced after old v5 backups go according to your retention settings.

[dadge]

OK, thanks. Time to start deleting some retention points!

[rhnb]

Interesting. We also use a DataDomain as the datastore for our backups but I didn't see any real difference when we upgraded to v6, and we still have a mix of both vbk's on the system. Be intersting to see if it does go down any when the final v5 vbk's are removed.
DD de-dupe is complex (think it's called variable length segment or something) so it's not a simple block by block comparison.

[Gostev]

I just wanted to correct Alexander, v6 backup format is not new. Internally, it is exactly the same as v5. What is different is that v6 now additionally uses small VBM file to store some metadata for fast access (which helps to speed up some operations). However, VBK is still self-sufficient and usable even if the VBM file is lost.

[dellock6]

Anton, thanks for correction, I was in doubt since as an example we use ExaGrid machines, they are certified for v5 but not already for v6, but anyway they were able to load v6 backups...

Luca.

[Daveyd]

I just upgraded to v6 and we currently use Exagrid. Are there any know issues with using an Exagrid CIFS share as a backup repository with v6? Also, should I be chosing the "Align backup file data blocks" or "Decompress backup data blocks before storing" (according to Exagrid, you should not be using compression at all)

Also, since if I am in the middle or starting a forward incremental chain, will the upgrade of v6 continue the incremental chain or will entirely new vbks be created?

[dellock6]

As I wrote elsewhere in these forums, ExaGrid guys told me v6 official support is lacking only in regards of the newly added Hyper-V support, but since v5 and v6 files have the same format, you are good to use ExaGrid on Veeam v6 for vSphere backups.

About advanced options, they stated alignment is not needed, decompression I think is needed if you are compressing the backup, but sound silly to me to compress and decompress while backing up in a lan environment, use this only if you are crossing a wan link and you have a veeam proxy on the other site.

Luca.

[Vitaliy S.]

according to Exagrid, you should not be using compression at all

Yes, that's right as dedupe vendors would typically recommend disabling compression, but keeping dedupe on. This way you will get dedupe ratio from storage perspective, at the cost of longer backup window (more data to transfer to storage over the network).