Theorical question about deduplication

[tntteam]

Hi,

I have some questions regarding how deduplication is supposed to work with Veeam.

Veeam deduplication is working on a "per job" basis.
I setup a backup job of 117 VMs (mixed windows 2008, 2012, linux'es).
I setup full each saturday, and start the backup job on thursday for the first time.
So I get a .vbk (=full) on thursday, plus 1 .vib on friday, then one .vbk on saturday.
Deduplication is on and set on "local target"
Compression is on, default settings

Why do my first full backup os 3.4TB and the second full backup that did happen on saturday (day+2) is 3.4TB too ?
The data can't have changed this much ?

What Am I missing ? Is the "per job" deduplication means "per run of each job" and deduplication doesn't work between each subsequent run of the same job?

Sorry it may be not very clear

[PTide]

Hi,

Why do my first full backup os 3.4TB and the second full backup that did happen on saturday (day+2) is 3.4TB too?

Deduplication takes place inside job, between VMs, which means that if there are many VMs with similar data (OS files, database files etc) in the same job then the resulting full backup file will be smaller than the total amount of data on all VMs due to deduplication applied to similar data. So, the statement

<...>deduplication doesn't work between each subsequent run of the same job<...>

is correct.

Thank you.

[dellock6]

Since you are running a "full" backup weekly, this backup has to have NO relation with previous chain and be independent. For this reason it does only deduplicate inside itself and doesn't look at blocks stored in previous restore points. Otherwise, you may look at forever-forward incremental or reversed incremental.

[tntteam]

THank you guys, I was misunderstanding the internal mechanics. I understand better why Veeam produced whitepapers about veeam+win2k12 builtin dedup.

Also when I see dedup 1.0x or 1.1x in backup results, I can conclude that deduplication is not worthy ?

[dellock6]

Again, it's based on the kind of operation that is in place.
those numbers usually comes out during an incremental run, but exactly because it's only extracting changed blocks compared to previous run, chances are those blocks are all new and unique, thus there's no other block similar to them to have dedupe between them

I know sometimes deduplication can be tricky to understand...