File under "System Volume Information" grows until disk full

[Brunok]

Hello,
i am new to Veeam, and have following question.

System:
Hyper-V Server 2012R2
4 virtual servers running 2012R2 (DC, SQL, TS, Veeam B&R)

The virtual disks are placed on the D:\ drive and using 500GB, 100 GB still free

When i create a full backup job and start this job, on the D:\System Volume Information\ a file is created and grows as the backup job runs. But this file grows until the D: drive is full, then, of course the VMs crashes (getting paused), and the job fails.
I think that this is a problem with my settings (deduplication ?)
Following Options are set for this job under "Storage" Advanced Settings"
Backup - Incremental, nothing else
Storage - Enable inline data dedup.. / Level optimal / Optimize Local target
Hyper-V Change block tracking enabled

What did i do wrong ?

Thank you for your help
Bruno

[Shestakov]

Hello Bruno and welcome to the Veeam community!

Could you elaborate on the issue? What type of error do you see under the "Action" section?
What is the repository and what is the total size of the VMs in the job?
It might be because you have less free space in a repository than the total size of VMs of the job.

If you just started to use Veeam B&R I would recommend you to backup the smallest VM just to see how it works.
Deduplication should not be a problem here. Thank you!

[Brunok]

Hello Nikita,
thanks for your input. Yes, indeed the total size in the Action section says: 06.09.2014 22:00:19 :: VM size: 1,5 TB (1,1 TB used)
The repository only has one 1TB. And its a kind a work around (it is a external usb drive, mounted as VHD on the backup server, as we do not have access to the NAS from the client at the moment).

>"It might be because you have less free space in a repository than the total size of VMs of the job."
Does that mean, that whenever i will create a "new" fullbackup, i have to be sure, that there is enough space free on the repository drive, otherwise it will fill up the (Hyper-V) local drives as long there is space (as happen on my server) ? I understand, that if on the repository is not enough space free, then the job fails, but it should not fill up other drives...

In the logs are following errors (for all 4 Servers the same..)

08.09.2014 22:04:30 :: Processing 'SRV1' Error: Client error: There is not enough space on the disk.

Failed to write data to the file [D:\Backups\Fullsave_all_VM\Fullsave_all_VM2014-09-08T220045.vib].
Failed to perform data backup.
An existing connection was forcibly closed by the remote host

08.09.2014 22:04:32 :: All VMs have been queued for processing -> (that is a green event)

08.09.2014 22:04:58 :: Backup location "D:\Backups" is getting low on free disk space (64,0 KB left of 931,5 GB).

and then my hyper-v machines getting paused...

Btw: i ran the job yesterday, without the "deduplication" flag. It worked ok, no problems. Under D:\System Volume Information are still created some files, but smaller in size, and does not fill up the drives.

Thanks
Bruno

[veremin]

The provided errors identify that the given repository was running out of space, thus, the backup jobs failed. As to System Volume folder, can you tell me what files are being created there during backup cycle (naming, extension, etc.)? Thanks.

[Brunok]

I've changed the job again to "inline data deduplication", started the job again to see the name. This file now is "28{3808876b-c176-4e48-b7ae-04046e6cc752}" without any extension, as fileype in windows explorer says "system file" - size is about 3072000 KB. The job runs ok now, even if i start it more than once - only the name of the created file change.. second job run creates "33{3808876b-c176-4e48-b7ae-04046e6cc752}". It seem the file is created when the backup process is at 8-10%, and is deleted when the job finishes.

Bruno

[veremin]

We're talking here about host disk, right? The files in question are Windows shadow copies created on the given volume. Thanks.

[Brunok]

yes, that was my presumption. But if the volume shadow copy fills up the disk until full, when the dedup is activated, can I prevent this somehow ? Thanks

[veremin]

I was trying to say that we're using snapshots created there during our backup activity. Once VMs are freezed, the snapshot of volume is taken. That's how our technology works.

But you seem to have not an ideal backup architecture in place. You have both VM disks and backups stored on one volume, right? So, what's actually happening here is that once the backup process is initiated, a newly created backup file, as well as, a volume snapshot start growing and occupying space on the volume, until it's filled completely. Moreover, the situation has nothing to do with deduplication settings. So, the best idea would be separate both activities, moving them to different disks.

Thanks.

[Brunok]

Thanks Vitali, in the meantime we've installed the servers and changed the repository to the clients NAS. Until now it looks good.. no problems.
Bruno

[veremin]

Yes, moving repository out of production host allows volume snapshots not to collide with growing backup files. Should other help be needed, don't hesitate to let us know. Thanks.