Snapshots and low disk space

Discussions specific to Microsoft Hyper-V hypervisor

Snapshots and low disk space

Veeam Logoby duhej » Thu May 11, 2017 4:25 am

Good afternoon. There is MS Server 2016 (2 pieces: VM source and VM receiver) and VBR U2 and configured in it backup and VM replication tasks from source Hyper-V to receiver Hyper-V. Periodically, randomly, on the HDD (on the VM source) with the VM, snapshots of the VM are created and then the free space on the disk with the VM ends and then the VMs do not work. This is totaly not necessary and creates a lot of problem problems for me. How to completely disable the creation of snapshots on the VM source, so that all backups and replicas are created ONLY to receiver Hyper-V?
duhej
Novice
 
Posts: 4
Liked: never
Joined: Thu May 11, 2017 4:09 am
Full Name: 111

Re: Snapshots and low disk space

Veeam Logoby Mike Resseler » Thu May 11, 2017 4:34 am

Hi @duhej,

First: Welcome to the forums!

I am not sure if I completely follow. Our replication technology is based on snapshot creation so we need to create a snapshot or we can't replicate the changes. That being said. If you are running out of disk space, that means that either it takes a long time for the replication which means there is a lot of changes that need to be processed or you are really low on disk space.

How much space do you have left? Are the checkpoints (new MSFT name for snapshots) placed on the same disk or redirected to another location?

Thanks
Mike
Mike Resseler
Veeam Software
 
Posts: 4226
Liked: 471 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Snapshots and low disk space

Veeam Logoby duhej » Thu May 11, 2017 2:47 pm

First: thanks for answer.
Second: i have source hyper-v and reciever hyper-v. I totaly do not need any snapshots or another changes to original VM vhdx files placed at source hyper-v. For snapshots or another changes i have reciever hyper-v. I have 1000 gb and I am running out of disk space, because there are a lot of snapshots of VM (at source hyper-v) placed on the same disk and this is the real problem.
duhej
Novice
 
Posts: 4
Liked: never
Joined: Thu May 11, 2017 4:09 am
Full Name: 111

Re: Snapshots and low disk space

Veeam Logoby Mike Resseler » Thu May 11, 2017 2:57 pm

You do need a temporary snapshot at the source side when the replication starts. Otherwise it is not possible to do the replication.

When you look here in our manual on how it works: https://helpcenter.veeam.com/docs/backu ... tml?ver=95

in point 8 you see: Veeam Backup & Replication requests Microsoft Hyper-V VSS to create a cohesive point-in-time copy of a VM. Microsoft Hyper-V VSS creates a volume snapshot or VM checkpoint, depending on the hypervisor version.

But that snapshot should be removed after the replication job is done
Mike
Mike Resseler
Veeam Software
 
Posts: 4226
Liked: 471 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Snapshots and low disk space

Veeam Logoby duhej » Sat May 13, 2017 7:16 am

Ok, so if there are a lot snapshots (chain of snapshots) at the same disk - something is wrong?
duhej
Novice
 
Posts: 4
Liked: never
Joined: Thu May 11, 2017 4:09 am
Full Name: 111

Re: Snapshots and low disk space

Veeam Logoby Mike Resseler » Mon May 15, 2017 10:36 am

Hi,

Yeah, I think something is wrong at that point in time. This could be for many reasons, and one of it could be Veeam (although when it is a chain, I doubt it but still...).

Now the issue with a chain of snapshots is that it could be dangerous to "remove" or "apply" them. So at this point in time, I would advise you to create a support call and let our engineers see if the issue is related to us or not, and at the same time they most probably will advise you what the best next step is to do (if it is not us that creates the chain at least...)

Please post the support ID here and the follow-up after working with support

Thanks
Mike
Mike Resseler
Veeam Software
 
Posts: 4226
Liked: 471 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Snapshots and low disk space

Veeam Logoby duhej » Fri May 19, 2017 4:27 am

thx
duhej
Novice
 
Posts: 4
Liked: never
Joined: Thu May 11, 2017 4:09 am
Full Name: 111

[MERGED] How much free space do I need at datastore?

Veeam Logoby Calby » Mon Feb 19, 2018 8:44 am

Hi,
I have notice that if the free space on the datastore disk goes under xxx then I can't backup with Veeam.
I have googled but I can't find out what the limit is?
How much free space do I need on the datastore?

Is it over 5% of the total harddrive space that needs to bee free?
Calby
Enthusiast
 
Posts: 25
Liked: never
Joined: Mon Jan 22, 2018 6:48 pm
Full Name: Robin Johansson

[MERGED] Re: How much free space do I need at datastore?

Veeam Logoby Mike Resseler » Mon Feb 19, 2018 8:55 am

Robin,

The reason of that free space is because we need to take a snapshot of the Hyper-V VM. At that moment, what happens is that the original VHD(X) gets "frozen" (read-only) and all writes are going to a differential file (AVHD(X)). This is the way the backup of VM's work in Hyper-V (and for that matter, more or less the same in VMware). If the datastore is low on space that file can't be placed there and also the merge (after the backup) won't be possible as it needs additional space also (working space, it gets free again after that)

You say 5%, but how much is that in GB? And what is your host OS? And lastly, what is the size off the VM in total?
Mike Resseler
Veeam Software
 
Posts: 4226
Liked: 471 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: [MERGED] Re: How much free space do I need at datastore?

Veeam Logoby Calby » Mon Feb 19, 2018 9:03 am

Mike Resseler wrote:Robin,

The reason of that free space is because we need to take a snapshot of the Hyper-V VM. At that moment, what happens is that the original VHD(X) gets "frozen" (read-only) and all writes are going to a differential file (AVHD(X)). This is the way the backup of VM's work in Hyper-V (and for that matter, more or less the same in VMware). If the datastore is low on space that file can't be placed there and also the merge (after the backup) won't be possible as it needs additional space also (working space, it gets free again after that)

You say 5%, but how much is that in GB? And what is your host OS? And lastly, what is the size off the VM in total?


Hi,
I'm running Hyper-V in Win2016.
The guest OS are running on a seperated SSD the system of the Guest OS. No issue there, but my storage disks that are attached to the VM are almost full and aldo they are not included in the backup (I have choosen to only backup system disk 0:0) it seem that they are making some issue.
I have excluded them from the backup in Veeam but they are still making some kind of issue.

First disk are 2,72GB with 94,6GB free (I can adjust this so I get more free space but then I need to know the limit, how much free space do I need)
The second disk 2,72GB with 125GB Free (Same here I can adjust it, but then I need to know how much free space I need)
Third disk is total 209GB (Don't know how much free space is have, but it's making some alerats)
Calby
Enthusiast
 
Posts: 25
Liked: never
Joined: Mon Jan 22, 2018 6:48 pm
Full Name: Robin Johansson

Re: Snapshots and low disk space

Veeam Logoby Mike Resseler » Mon Feb 19, 2018 9:07 am

Hmm. I would rather say you have space enough for those sizes. But, I might be wrong here but your VMs OS disk is only 2.72 GB? that seems low if it is a windows VM?

The disk free space I cannot calculate because it depends on how much churn there is (and the growth of the AVHDX because of that). If it is a very busy VM during the backup, it will be a big AVHDX file (could even be bigger than the original one if many changes occur).

How much is the total size of those datastores?
Mike Resseler
Veeam Software
 
Posts: 4226
Liked: 471 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Snapshots and low disk space

Veeam Logoby Calby » Mon Feb 19, 2018 9:14 am

Mike Resseler wrote:Hmm. I would rather say you have space enough for those sizes. But, I might be wrong here but your VMs OS disk is only 2.72 GB? that seems low if it is a windows VM?

The disk free space I cannot calculate because it depends on how much churn there is (and the growth of the AVHDX because of that). If it is a very busy VM during the backup, it will be a big AVHDX file (could even be bigger than the original one if many changes occur).

How much is the total size of those datastores?


Sorry I do mean TB not GB sorry for that, the total space is in TB and the space left is in GB.
Beside the 209GB disk that one is 209GB in total.

The AVHDX is big on this diskes, they are over 2TB in size.

Production drive E:\ is getting low on free space (125,3 GB left), and may run out of free disk space completely due to open snapshots.
Production drive K:\ is getting low on free space (96,5 GB left), and may run out of free disk space completely due to open snapshots.


AVHDX sizes:
K: 2,61 TB
E: 2,59 TB
Calby
Enthusiast
 
Posts: 25
Liked: never
Joined: Mon Jan 22, 2018 6:48 pm
Full Name: Robin Johansson

Re: Snapshots and low disk space

Veeam Logoby Mike Resseler » Mon Feb 19, 2018 9:22 am

That means you are already running on snapshots for some time I think. Can you check on Hyper-V manager to see if there are open snapshots? Potentially multiple?
Mike Resseler
Veeam Software
 
Posts: 4226
Liked: 471 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Snapshots and low disk space

Veeam Logoby Calby » Mon Feb 19, 2018 9:33 am

Mike Resseler wrote:That means you are already running on snapshots for some time I think. Can you check on Hyper-V manager to see if there are open snapshots? Potentially multiple?


No, no other snapshots.
And I have tried to run Veeam again.
Then I get: 2018-02-19 10:30:43 :: Processing Media Error: Skipping VM processing due to insufficient free disk space on drive E:\.

I guess that I get this error now because I have increased the "Skip VM processing when free disk space are below" to 5% again as I did try to turn it down to 1% before.
If I have it on 1% I do get this kind of warnings as I did write about above.

But is the free diskspace limit 5% on the datastore disks?

Edit:
My guess is that I did get the first warning because when Veeam are opening a snapshot it get's out of free space - this is based on that when I do increase the "Skip VM processing when free disk space are below" to 5% I do get a error because of low diskspace and if I lower or turn the "Skip VM processing when free disk space are below" of then I get a warning.
Because I don't have any other kind of snapshot opened.

So if I do increse the free diskspace this should be solved, but as I don't know the free space limit it's kind of hard to do =/
Can you point me to the right direction?
Calby
Enthusiast
 
Posts: 25
Liked: never
Joined: Mon Jan 22, 2018 6:48 pm
Full Name: Robin Johansson

Re: Snapshots and low disk space

Veeam Logoby Mike Resseler » Mon Feb 19, 2018 9:41 am

From what I read here, I think it is indeed on the disks that hold the VMs.

I would suggest at this point in time to open a support call so our engineers can help you with (at least I believe this will be the way to go if I read everything correctly) merging the AVHDX file (unless that AVHDX is from a cluster disk, which unfortunately also uses AVHDX as extension) and then work further from there. As long as we don't have the space, we can't take that backup. If you run out of it... Well, you want to avoid that :-)

I think it is better that the engineers look at it with the logs and also maybe with a support session because if I would give you advice to do a merge here, without actually seeing what is going on, I might give you the wrong advice and you could loose production data which I don't want to happen :-)

Please post the support case ID here and the outcome after investigation/ working with the engineers

Thanks
Mike
Mike Resseler
Veeam Software
 
Posts: 4226
Liked: 471 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Next

Return to Microsoft Hyper-V



Who is online

Users browsing this forum: jay.cheon and 18 guests