Production Drive is getting low on free space - HyperV - CSV

Hyper-V specific discussions

Production Drive is getting low on free space - HyperV - CSV

Veeam Logoby drubs » Fri Apr 29, 2016 12:42 pm

Hi all,

So setup is as follows;
HyperV
-3 Node Hyper-V Cluster (Server 2012 R2)
-VMs stored over 2 CSVs
-Underlying FibreChannel Storage (no access to Hardware VSS, so using SoftwareVSS)

Veeam Backup and Replication v9
-Physical Server
-Direct Attached Storage (SAS to JBOD)
-Storage repository provided by Storage Spaces (6PhysicalDisk, 1PhysicalDiskHotspare, 2wayMirror, 3Column)

Network
-Everything is talking over 10GbE

Backup Jobs and Scheduling
-50VMs between multiple tenants

And error message I'm receiving;
"Production drive 'C:\ClusterStorage\Volume1' is getting low on free space (2.9 GB left), and may run out of free disk space completely due to open snapshots."

Only occuring on some a few days during the last week. First time it happened, I came into work with a large number of VMs(All on C:\ClusterStorage\Volume1), in a paused state due to insufficient storage. Not fun.

C:\ClusterStorage\Volume1 normally has at least 1.2TB of free space. But during backups only seems to have 20GB freespace. Finding it hard to believe that snapshots are using that much space.

There are 35 VMs that live on CSV1 and that are backed up, and they total roughly 3.3TB storage used(both dynamic disks and fixed disks). These 35 VMs are seperated into I think around 5 or 6 jobs

Questions
Am I underestimating the size that the Snapshot will be?
Or is there a way to tell the size and anticipate how much is required?
Is there a chance that snapshots are being left open permanently?

Sorry i know there a lot of information, but just any help in understanding the issue would be awesome!

Cheers
Chris
drubs
Novice
 
Posts: 4
Liked: never
Joined: Tue Jan 12, 2016 4:51 am

Re: Production Drive is getting low on free space - HyperV -

Veeam Logoby foggy » Fri Apr 29, 2016 3:13 pm

Snapshot size depends on how active VM is and the time it runs off the snapshot. I'd check for existing snapshots (both VM-level and volume-level). For volume-level you can use vssadmin list shadowstorage and vssadmin list shadows (or DISKSHADOW> list shadows all) commands.
foggy
Veeam Software
 
Posts: 14742
Liked: 1079 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Production Drive is getting low on free space - HyperV -

Veeam Logoby drubs » Sat Apr 30, 2016 2:45 am

Thanks for the suggestions Foggy, that put me in the right direction to clearing up the issue.

Tried the above VSSadmin commands, but nothing showed up. Researched a bit more as to why not and was led to a suggestion in another forum, to move the CSV ownership to another node. Once i did this the expected free storage was accurate again (1.2TB).

Not really a fix, and I'm sure there is still some underlying behavior that I don't quite understand. But for now I will keep a closer eye on it, and if I find anything post back here.

Cheers
Chris
drubs
Novice
 
Posts: 4
Liked: never
Joined: Tue Jan 12, 2016 4:51 am

Re: Production Drive is getting low on free space - HyperV -

Veeam Logoby v.Eremin » Mon May 02, 2016 9:05 am

As it's rather hard to find out the root cause of the experienced behaviour through forum correspondence, it might stand to reason to open a support ticket and let them investigate it directly, making a definite statements after logs analysis, etc. Thanks.
v.Eremin
Veeam Software
 
Posts: 13266
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Production Drive is getting low on free space - HyperV -

Veeam Logoby nmdange » Mon May 02, 2016 3:41 pm 2 people like this post

If changing the CSV owner resulted in space getting freed up, then it is likely due to the System Volume Information folder growing unexpectedly. Make sure you have the updates I mentioned in this post: microsoft-hyper-v-f25/csv-system-volume-information-size-growing-t34478.html
nmdange
Expert
 
Posts: 193
Liked: 57 times
Joined: Thu Aug 20, 2015 9:30 pm

Re: Production Drive is getting low on free space - HyperV -

Veeam Logoby pantos21 » Mon Sep 26, 2016 6:29 am

Hi all,

I have the exact same issue and when i change the csv owner the space i freed up. But i have windows 2012 and not 2012 R2 that the recomened pathces works. I canot find these patches for windows 2012

regards
pantos
pantos21
Novice
 
Posts: 3
Liked: never
Joined: Thu Jun 26, 2014 9:51 am
Full Name: pantos

Re: Production Drive is getting low on free space - HyperV -

Veeam Logoby foggy » Mon Sep 26, 2016 4:34 pm

You can open a case with Microsoft to check if similar patches exist for earlier version.
foggy
Veeam Software
 
Posts: 14742
Liked: 1079 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson


Return to Microsoft Hyper-V



Who is online

Users browsing this forum: No registered users and 5 guests