Comprehensive data protection for all workloads
aharvey
Novice
Posts: 8
Liked: 1 time
Joined: Apr 08, 2011 2:47 pm
Full Name: Aaron Harvey
Contact:

[MERGED] deduped VM backup size numbers not adding up

Post by aharvey »

I am misunderstanding or overlooking something on my Veeam backups, but either way, could use some help. Here is the situation. I have a Server 2012 VM that is our main file server. It has 3 local volumes on 3 vmdk files. C: is the OS, J: and K: are 2 data drives that have 2012 data deduplication enabled on them. Here are the numbers from the OS:

Drive Size / Used Space / Dedup Savings
C: 50 GB / 15 GB / N/A
J: 500 GB / 167 GB / 42 GB
K: 750 GB / 497 GB / 119 GB

Totals 1.3 TB / 679 GB / 161 GB

I ran a Veeam full backup of the server with in-line data deduplication enabled. The Job details show the VM size: 1.3 TB (1.0 TB used). I'm not sure where it is getting the 1.0 TB used value?

The main issue is, the backup file (.vbk) is 987 GB. Here are the numbers from the job:
Hard Disk 1 (50.0 GB) 17.1 GB read
Hard Disk 2 (500.0 GB) 323.1 GB read
Hard Disk 3 (750.0 GB) 723.1 GB read

Processed: 1.0 TB (100%)
Read: 1.0 TB
Transferred: 986.7 GB

Even if the backup re-hydrated the files before backing up (I don't think it is designed that way) that is only 840 GB of uncompressed raw data. How can the backup file be 147 GB larger than the raw data. I know there are vmx files etc. backed up, but those take up minimal space. How is the data read from each hard disk greater (by far) than the amount of data on the disk (whether you factor in deduplication or not)?

I would expect the vbk file to be close to 679 GB, or perhaps even a bit smaller with in-line dedup/compression enabled for the job.

Can someone help shed some light on what's happening here?

Thanks
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Backup size larger than

Post by Vitaliy S. »

Aaron,

Most likely you had some data written to these disks before and this made VMware CBT treat these blocks as blocks containing data. Please see this topic for more info on the observed behavior.

Let me know if that helps!
rct
Enthusiast
Posts: 35
Liked: 1 time
Joined: Sep 29, 2011 6:07 pm
Contact:

Re: Backup size larger than

Post by rct »

See this topic: http://www.mylesgray.com/infrastructure ... oned-vmdk/ (if you just sdelete your volumes the vmdk will be full-sized but the deduplicated backup will be smaller)

I've done the "same" on my linux box and using dedup&compression in veeam my backup size dropped of 1.5TB!!!
Elefanten
Influencer
Posts: 20
Liked: 1 time
Joined: Jul 06, 2009 10:56 am
Contact:

[MERGED] : Backup Size bigger than actual size of disks

Post by Elefanten »

Hello

My goal is to do a backup job from China to Norway over a 0.5Mbit line with accelerator usage.

I am setting up New Backup Copy Job and to my surprise when i choose the virtual machine its size is 473 GB when the VM has total 2 VM disk (OS 40GB and Data 200 GB).
But OS disk has only used 14GB and Data 63 GB.

I know there is something about sdelete, but this makes me nervous to use ?
Question.
How do i safely use sdelete if this can solve it ?
what is the difference on c and z switches ?

Regards
Elefanten
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup size larger than

Post by veremin »

what is the difference on c and z switches ?
It depends on sdelete version. In sDelete 1.6 (and newer):

Code: Select all

-z Zero free space
-c Clean free space
Thanks.
jgl75
Influencer
Posts: 24
Liked: 5 times
Joined: Oct 02, 2013 6:23 am
Contact:

[MERGED] Windows Server 2012 R2 Deduplicated VM and volume o

Post by jgl75 »

Hello !

I've recently setup a new infrastructure with the following :

- 1 ESXi 5.5 Host with a 7To local datastore (ML350 Gen 8 with 8*1.2To SAS in RAID5)
- 2 VMs with 3To of of provisioned HDD space.

Both VM are Windows Server 2012 R2 with deduplication activated.
The deduplication rate seems good (47 & 35%) and I've seen massive delta backup when de dedup process occurred (which seems normal to me).

Here's my question :

Should the first active full backup size (after dedupe) reflect the real size used on the VMDK ?
I've 1.4To of free space on the first server and my backup file size for this server is 2.33To (while used space should be around 1.6To without talking about compression)...

My problem is that I'm writing my backup files on tape (LTO6) every week and I'm afraid that the backup file size will grow above 2.5To at some point...
My compression is set to High already.

Thanks in advance for your help !

Jean
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup size larger than

Post by foggy » 1 person likes this post

jgl75 wrote:Should the first active full backup size (after dedupe) reflect the real size used on the VMDK ?
Used by VMDK (not reported as used by the guest OS but effectively used on VMFS storage), since Veeam B&R is an image-based block-level solution.
jgl75
Influencer
Posts: 24
Liked: 5 times
Joined: Oct 02, 2013 6:23 am
Contact:

Re: Backup size larger than

Post by jgl75 »

Thanks Foggy for always giving a very fast answer !

I was not searching in the right direction (on the forums), sorry...
avrham_g
Enthusiast
Posts: 34
Liked: 1 time
Joined: Dec 29, 2011 9:41 pm
Full Name: Avi
Contact:

[MERGED] strange backup size

Post by avrham_g »

hi,
i use B&R 7.0.0.833
i backup SBS2008 with reversed incremental the total used space on the SBS for C and D drive are 576GB.
why when i run the first full backup of the SBS the VBK file is 716GB?

THX
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup size larger than

Post by foggy »

Please review the thread above, should answer your question. Thanks.
acrismatic
Influencer
Posts: 13
Liked: never
Joined: Jan 13, 2015 12:51 pm
Contact:

[MERGED] Difference disk size

Post by acrismatic »

Hi all,

I have a question about how Veeam works with Exchange.
We have a 7.75 TB Volume for the backup jobs of Exchange. This volume is added as iSCSi target to a w2008R2 server. When I check the used and free space on this volume under windows, I can see:
used space: 1.79 TB
free space: 5.95 TB

But when I check the same on our Synolgy disk station we have a significative difference:
used space: 3.89 TB
free space: 3.9 TB

Do you know what is happening? Why we have under 2TB of difference?

In other Volumes where we keep other VM's backup, free and used space are correct on windows and on disk station.

Thanks in advance.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup size larger than

Post by foggy »

Please review the thread above. Remember that NTFS does not zero out disk blocks when file is deleted, but rather just marks them empty (though they still contain data and are backed up by image-level solutions). You need to use sdelete or similar tools to really turn unused disk space into white space.
jamerson
Veteran
Posts: 366
Liked: 24 times
Joined: May 01, 2013 9:54 pm
Full Name: Julien
Contact:

[MERGED] Backup technique

Post by jamerson »

Dear all,
i have a questions regarding disk size and backup size.

i have 1TB disk lazy zero on VMWARE, on the disk i have 200GB data.
every backup dat VEEAM does, its backup 1TB.
is it possible to escape the empty storage and backup just the 200GB data ?

thank you
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup size larger than

Post by foggy »

Julien, have you tried to zero the free space inside guest OS?
jamerson
Veteran
Posts: 366
Liked: 24 times
Joined: May 01, 2013 9:54 pm
Full Name: Julien
Contact:

Re: Backup size larger than

Post by jamerson »

foggy wrote:Julien, have you tried to zero the free space inside guest OS?
do you mean on my File server ? how i am supposed to do that ?
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Backup size larger than

Post by PTide »

every backup dat VEEAM does, its backup 1TB.
Do you mean that every time you run your backup it's still 1TB in size no matter if it's incremental or full? Btw, what OS do you run - is it Linux of Windows?
jamerson
Veteran
Posts: 366
Liked: 24 times
Joined: May 01, 2013 9:54 pm
Full Name: Julien
Contact:

Re: Backup size larger than

Post by jamerson »

PTide wrote:Do you mean that every time you run your backup it's still 1TB in size no matter if it's incremental or full? Btw, what OS do you run - is it Linux of Windows?
yes this what I mean. all the backup I am using is incremental
its backup the disk size not the files size.
I am on OS 2012R2 64 bit
Thank you
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Backup size larger than

Post by PTide »

So, you have a VM with a 1 TB .vmdk and only 1 x 200 GB guest OS (Windows 2012R2) volume atop that .vmdk? Are the remaining 800 GB allocated? Also, do you have Windows deduplication enabled? If so then what is your backup schedule and what is your deduplication schedule?

Thank you.
jamerson
Veteran
Posts: 366
Liked: 24 times
Joined: May 01, 2013 9:54 pm
Full Name: Julien
Contact:

Re: Backup size larger than

Post by jamerson »

PTide wrote:So, you have a VM with a 1 TB .vmdk and only 1 x 200 GB guest OS (Windows 2012R2) volume atop that .vmdk? Are the remaining 800 GB allocated? Also, do you have Windows deduplication enabled? If so then what is your backup schedule and what is your deduplication schedule?

Thank you.
Thank you for your answer,
yes I do have File serve with two disks
disk 1 + 40GB ssd which is running the OS
disk 2 1TB " 200GB data and 800 GB free.
Windows Duplication is not enabled and my schedule is one a day and keep the restore points for 5 days .

thank you
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup size larger than

Post by foggy »

How do you tell the entire 1TB is backed up? Do you see the huge read/transferred counters values each time? Have you reviewed the thread above for some hints regarding the observed behavior?
lightsout
Expert
Posts: 227
Liked: 62 times
Joined: Apr 10, 2014 4:13 pm
Contact:

Re: Backup size larger than

Post by lightsout »

So here is how I handled this problem. I guess this will be "fixed" in v9, but here is how I work in v8.

My core script is this vbscript, which will run sdelete on all drives on the server. It assumes the sdelete.exe is in the same directory as the vbscript file:

Code: Select all

Set WshShell = WScript.CreateObject("WScript.Shell")

Dim fso, d, dc
Set fso = CreateObject("Scripting.FileSystemObject")
Set dc = fso.Drives

strSFN = wscript.scriptfullname
strSFN = Left(strSFN,InstrRev(strSFN,"\"))

WshShell.RegWrite "HKCU\Software\Sysinternals\", 0, "REG_SZ"
WshShell.RegWrite "HKCU\Software\Sysinternals\SDelete\", 0, "REG_SZ"
WshShell.RegWrite "HKCU\Software\Sysinternals\SDelete\EulaAccepted", 1, "REG_DWORD"

For Each d in dc
If d.DriveType = 2 Then
  wscript.echo "Zeroing: " & d
  wshShell.Run strSFN & "sdelete -z " & d, 1, TRUE
End If
Next

Set WshShell = Nothing
I run this on all servers using a scheduled task, that runs once a month. It has a randomization period of 24hrs, over the weekend, as to not overload the SAN system. This will make Veeam spend a lot of time processing the backups after this runs, but I'm OK with that, but your mileage may vary!

I've also found running a file system defrag can help, as sdelete can be VERY slow if the file system is heavily fragmented. So I run that just before sdelete runs.

I can provide further detail, if people are interested.
Edage
Service Provider
Posts: 6
Liked: never
Joined: Apr 29, 2014 9:01 am
Full Name: Patiphon Techacheewapong
Contact:

[MERGED] Veeam read vm oversize

Post by Edage »

Let's me explain about my situation first.

I had implemented Veeam for my customer and it work successfully, but after a while they contact me that it look like veeam read 1 vm oversize from what it used currently.

They send information about that vm, it have Thin provisioning 2.4 TB and used around 1.85 TB, but when Veeam backup it read this vm 2.2 TB.

So they concern that why Veeam read vm more capacity than it have. And they already check vm disk on window explorer also confirm that it used around 1.8 TB.

For me Veeam never have this problem before so I expected that Veeam read it correctly, but how should I prove it for them?
lando_uk
Veteran
Posts: 371
Liked: 32 times
Joined: Oct 17, 2013 10:02 am
Full Name: Mark
Location: UK
Contact:

Re: Veeam read vm oversize

Post by lando_uk »

If you're not doing occasional Full backups, you need to defrag and compact your jobs occasionally (or run an active Full)
https://helpcenter.veeam.com/docs/backu ... tml?ver=95
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup size larger than

Post by foggy »

Hi Patiphon, please review this thread for details. Basically, this is explained by NTFS design and the fact the data is being deleted inside guest OS.
Edage
Service Provider
Posts: 6
Liked: never
Joined: Apr 29, 2014 9:01 am
Full Name: Patiphon Techacheewapong
Contact:

Re: Backup size larger than

Post by Edage »

Thank you for your help foggy, I will review if it can help me solve my problem.
Post Reply

Who is online

Users browsing this forum: Bing [Bot], Google [Bot] and 153 guests