-
- Influencer
- Posts: 12
- Liked: 4 times
- Joined: Nov 23, 2020 9:57 am
- Full Name: Emilio González González
- Contact:
Backup far away bigger than source data. Any idea to solve it?
I'm a bit desperate.
I have a VM with one 1.68 TB virtual disk (the disk is bigger but 1.68 TB is the space used).
I back up the VM and obtain a "full backup file" of 2.7 TB. That's 1 TB bigger than source data!!
True, my source data is already compressed, then I'm not expecting any gain, in fact, I have chosen no compression at all (just in case I have tested with compression too and the result is similar). I can deal with some overhead for indexes, metadata, or whatever but for 1,68 TB the backup has added 1 TB of information to the original!!
Finally, I must point out that the "source already compressed data" is just a veeam NAS backup (with "compression level: High"). That's the reason I was expecting no gain (but no loss).
Resume.
I Backup a NAS with a high compression level option: from 2,1 TB original NAS user files, the veean backup system was able to reduce it to 1,7 TB
I Backup the VM with the previous backup of 1.7 TB (27,500 files) and obtained a unique file of 2.7 TB (even more than the 2.1 TB original uncompressed data in the NAS). I could easily understand that going from one file to thousands there is a loss in filesystem space because some are small, block size, etc. but this is just the opposite case.
Does anyone have an explanation?
And more important Is there any way to backup my VM that contains my 1.7 TB NAS Backup without such a loss?
In other words, Is there any way to "join all these files I have in my disk" in one with veeam backup format (like a tar in Linux or a zip with no compression in windows) but without adding 1 TB to the result?
Best regards
PD. You may wonder why I do this. This is another issue, my question here is how can I backup a disk with whatever information in it (files in the end, even if highly compressed) and obtain a file similar in size, or only slightly bigger than the original.
I have a VM with one 1.68 TB virtual disk (the disk is bigger but 1.68 TB is the space used).
I back up the VM and obtain a "full backup file" of 2.7 TB. That's 1 TB bigger than source data!!
True, my source data is already compressed, then I'm not expecting any gain, in fact, I have chosen no compression at all (just in case I have tested with compression too and the result is similar). I can deal with some overhead for indexes, metadata, or whatever but for 1,68 TB the backup has added 1 TB of information to the original!!
Finally, I must point out that the "source already compressed data" is just a veeam NAS backup (with "compression level: High"). That's the reason I was expecting no gain (but no loss).
Resume.
I Backup a NAS with a high compression level option: from 2,1 TB original NAS user files, the veean backup system was able to reduce it to 1,7 TB
I Backup the VM with the previous backup of 1.7 TB (27,500 files) and obtained a unique file of 2.7 TB (even more than the 2.1 TB original uncompressed data in the NAS). I could easily understand that going from one file to thousands there is a loss in filesystem space because some are small, block size, etc. but this is just the opposite case.
Does anyone have an explanation?
And more important Is there any way to backup my VM that contains my 1.7 TB NAS Backup without such a loss?
In other words, Is there any way to "join all these files I have in my disk" in one with veeam backup format (like a tar in Linux or a zip with no compression in windows) but without adding 1 TB to the result?
Best regards
PD. You may wonder why I do this. This is another issue, my question here is how can I backup a disk with whatever information in it (files in the end, even if highly compressed) and obtain a file similar in size, or only slightly bigger than the original.
-
- Chief Product Officer
- Posts: 31805
- Liked: 7299 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Backup far away bigger than source data. Any idea to solve it?
I assume the file system for the problematic disk is not NTFS?
-
- Influencer
- Posts: 12
- Liked: 4 times
- Joined: Nov 23, 2020 9:57 am
- Full Name: Emilio González González
- Contact:
Re: Backup far away bigger than source data. Any idea to solve it?
Both, the VM disk where the NAS backup goes and the repository where this VM disk backup goes are ReFS
-
- Chief Product Officer
- Posts: 31805
- Liked: 7299 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Backup far away bigger than source data. Any idea to solve it?
OK, just as I thought. Thing is, most file systems don't zero out disk blocks when you delete files, so with image-level backups you are backing up those "dirty" blocks belonging to those now-deleted files. Using a tool like sdelete to zero out unused parts of the disk should help. Thanks!
-
- Influencer
- Posts: 12
- Liked: 4 times
- Joined: Nov 23, 2020 9:57 am
- Full Name: Emilio González González
- Contact:
Re: Backup far away bigger than source data. Any idea to solve it?
Thanks, just to know. Can anyone confirm that veeam stores non used blocks in the backups? That is that if I have a 1 TB disk full of data and I delete all the data (empty disk). Will veeam backup all this data? and in case I use no compression at all, achieve a 1TB backup from an empty disk?
-
- Veeam Software
- Posts: 21138
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Backup far away bigger than source data. Any idea to solve it?
Basically, yes - any image-level backup solution does so. For NTFS though, Veeam has BitLooker feature for deleted blocks removal.
-
- Influencer
- Posts: 12
- Liked: 4 times
- Joined: Nov 23, 2020 9:57 am
- Full Name: Emilio González González
- Contact:
Re: Backup far away bigger than source data. Any idea to solve it?
But Even when we use the recommended setting in Storage->Adavanced Options "Exclude Deleted File blocks" and "Exclude Swap file Blocks"?
I was assuming that in that case, the only backup data was the real data, not the empty space.
I was assuming that in that case, the only backup data was the real data, not the empty space.
-
- Veeam Software
- Posts: 21138
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Backup far away bigger than source data. Any idea to solve it?
Exclude deleted file blocks option is right the functionality I'm talking about (BitLooker) - as mentioned, it is applicable to NTFS only. That said I find your setup sub-optimal. Storing NAS backup (or any other backup) on a VM is not the best practice; while to have a secondary copy of your backup, you can use the corresponding functionality in the NAS backup job itself.
-
- Influencer
- Posts: 12
- Liked: 4 times
- Joined: Nov 23, 2020 9:57 am
- Full Name: Emilio González González
- Contact:
Re: Backup far away bigger than source data. Any idea to solve it?
Foggy, you are right, but there is a reason for doing this strange thing. Storing NAS backup on a VM and doing a backup of this VM in a SOBR is just a workaround to be able to upload the backup to the cloud (Azure storage) as VBR does not support NAS backup offload to cloud storage vía SOBR. But offloading VM backups are supported, then this is what we are doing. Till the moment no one has presented me with a solution to upload to Azure (or S3) a veeam NAS backup. If you give me a better idea you will be my hero!
-
- Influencer
- Posts: 12
- Liked: 4 times
- Joined: Nov 23, 2020 9:57 am
- Full Name: Emilio González González
- Contact:
Re: Backup far away bigger than source data. Any idea to solve it?
Finally solved. Thanks to all of you. After your key answers and a lot of reading veeam doc my back-up is the right size. My problem was using a ReFS repository for the first backup (the NAS to the VM), which as mentioned by some of you has no option to avoid storing delete files or empty space. After moving to an NTFS repository the backup is no bigger than the source data (the SOBR continues to be ReFS). I prefer this to "zeroing" empty space with tools like sdelete.
Who is online
Users browsing this forum: sally123 and 76 guests