Host-based backup of Microsoft Hyper-V VMs.
Post Reply
Brunok
Enthusiast
Posts: 36
Liked: 6 times
Joined: Sep 02, 2014 7:16 am
Full Name: Bruno
Contact:

File under "System Volume Information" grows until disk full

Post by Brunok »

Hello,
i am new to Veeam, and have following question.

System:
Hyper-V Server 2012R2
4 virtual servers running 2012R2 (DC, SQL, TS, Veeam B&R)

The virtual disks are placed on the D:\ drive and using 500GB, 100 GB still free

When i create a full backup job and start this job, on the D:\System Volume Information\ a file is created and grows as the backup job runs. But this file grows until the D: drive is full, then, of course the VMs crashes (getting paused), and the job fails.
I think that this is a problem with my settings (deduplication ?)
Following Options are set for this job under "Storage" Advanced Settings"
Backup - Incremental, nothing else
Storage - Enable inline data dedup.. / Level optimal / Optimize Local target
Hyper-V Change block tracking enabled

What did i do wrong ?

Thank you for your help
Bruno
Shestakov
Veteran
Posts: 7328
Liked: 781 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: File under "System Volume Information" grows until disk

Post by Shestakov »

Hello Bruno and welcome to the Veeam community!

Could you elaborate on the issue? What type of error do you see under the "Action" section?
What is the repository and what is the total size of the VMs in the job?
It might be because you have less free space in a repository than the total size of VMs of the job.

If you just started to use Veeam B&R I would recommend you to backup the smallest VM just to see how it works.
Deduplication should not be a problem here. Thank you!
Brunok
Enthusiast
Posts: 36
Liked: 6 times
Joined: Sep 02, 2014 7:16 am
Full Name: Bruno
Contact:

Re: File under "System Volume Information" grows until disk

Post by Brunok »

Hello Nikita,
thanks for your input. Yes, indeed the total size in the Action section says: 06.09.2014 22:00:19 :: VM size: 1,5 TB (1,1 TB used)
The repository only has one 1TB. And its a kind a work around (it is a external usb drive, mounted as VHD on the backup server, as we do not have access to the NAS from the client at the moment).

>"It might be because you have less free space in a repository than the total size of VMs of the job."
Does that mean, that whenever i will create a "new" fullbackup, i have to be sure, that there is enough space free on the repository drive, otherwise it will fill up the (Hyper-V) local drives as long there is space (as happen on my server) ? I understand, that if on the repository is not enough space free, then the job fails, but it should not fill up other drives...

In the logs are following errors (for all 4 Servers the same..)

08.09.2014 22:04:30 :: Processing 'SRV1' Error: Client error: There is not enough space on the disk.

Failed to write data to the file [D:\Backups\Fullsave_all_VM\Fullsave_all_VM2014-09-08T220045.vib].
Failed to perform data backup.
An existing connection was forcibly closed by the remote host

08.09.2014 22:04:32 :: All VMs have been queued for processing -> (that is a green event)

08.09.2014 22:04:58 :: Backup location "D:\Backups" is getting low on free disk space (64,0 KB left of 931,5 GB).

and then my hyper-v machines getting paused...

Btw: i ran the job yesterday, without the "deduplication" flag. It worked ok, no problems. Under D:\System Volume Information are still created some files, but smaller in size, and does not fill up the drives.

Thanks
Bruno
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: File under "System Volume Information" grows until disk

Post by veremin »

The provided errors identify that the given repository was running out of space, thus, the backup jobs failed. As to System Volume folder, can you tell me what files are being created there during backup cycle (naming, extension, etc.)? Thanks.
Brunok
Enthusiast
Posts: 36
Liked: 6 times
Joined: Sep 02, 2014 7:16 am
Full Name: Bruno
Contact:

Re: File under "System Volume Information" grows until disk

Post by Brunok »

I've changed the job again to "inline data deduplication", started the job again to see the name. This file now is "28{3808876b-c176-4e48-b7ae-04046e6cc752}" without any extension, as fileype in windows explorer says "system file" - size is about 3072000 KB. The job runs ok now, even if i start it more than once - only the name of the created file change.. second job run creates "33{3808876b-c176-4e48-b7ae-04046e6cc752}". It seem the file is created when the backup process is at 8-10%, and is deleted when the job finishes.

Bruno
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: File under "System Volume Information" grows until disk

Post by veremin »

We're talking here about host disk, right? The files in question are Windows shadow copies created on the given volume. Thanks.
Brunok
Enthusiast
Posts: 36
Liked: 6 times
Joined: Sep 02, 2014 7:16 am
Full Name: Bruno
Contact:

Re: File under "System Volume Information" grows until disk

Post by Brunok »

yes, that was my presumption. But if the volume shadow copy fills up the disk until full, when the dedup is activated, can I prevent this somehow ? Thanks
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: File under "System Volume Information" grows until disk

Post by veremin »

I was trying to say that we're using snapshots created there during our backup activity. Once VMs are freezed, the snapshot of volume is taken. That's how our technology works.

But you seem to have not an ideal backup architecture in place. You have both VM disks and backups stored on one volume, right? So, what's actually happening here is that once the backup process is initiated, a newly created backup file, as well as, a volume snapshot start growing and occupying space on the volume, until it's filled completely. Moreover, the situation has nothing to do with deduplication settings. So, the best idea would be separate both activities, moving them to different disks.

Thanks.
Brunok
Enthusiast
Posts: 36
Liked: 6 times
Joined: Sep 02, 2014 7:16 am
Full Name: Bruno
Contact:

Re: File under "System Volume Information" grows until disk

Post by Brunok » 1 person likes this post

Thanks Vitali, in the meantime we've installed the servers and changed the repository to the clients NAS. Until now it looks good.. no problems.
Bruno
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: File under "System Volume Information" grows until disk

Post by veremin »

Yes, moving repository out of production host allows volume snapshots not to collide with growing backup files. Should other help be needed, don't hesitate to let us know. Thanks.
Post Reply

Who is online

Users browsing this forum: No registered users and 21 guests