Disk Space Issue

[EmptySet]

Hello,

I'm new to the world of Veeam and am trying to get a up to speed on an issue I'm seeing; during one of our incremental backups I noticed a failure due to low disk space. This disk in question is 9TB, and when I look at the usage in Windows Explorer (right click backup folders and looked at size field), it only shows that ~5TB is used. 'Size on disk' is much smaller at ~2.2TB, which I'm assuming is due to some compression. If I just look at the drive in Windows though, it says I only have ~300GB free. I'm confused at how there's only ~5.5TB in the folders but it thinks there's almost 9TB in use.

As I'm unfamiliar with Veeam, I'm wondering if when an old backup is deleted, that the space isn't actually being released?

Any help would appreciated.

Thanks.

[Mike Resseler]

Hi,

I doubt this has something to do with Veeam to be honest.

What OS and filesystem are you running? If windows explorer says there is only 300 GB free, that probably means you have files on there which you are missing. Could be hidden files or something. Can you open Windows Explorer, make sure you see hidden files and check if there are some hidden somewhere on that disk?

[EmptySet]

Hidden files are enabled and when I select all folders and look at the size, it's at ~5TB in use.

[foggy]

Do you also have protected operating system files shown? System Restore folder could use the space, for example. Could be alternate data streams as well or something else, but definitely not Veeam-related.

[csinetops]

Download "WinDirStat", install, right click and run as "Administrator". Select your Veeam repository drive in the GUI and let it crunch. That will show you any files you are missing in explorer. I'm on point with foggy, chances are you don't have permissions to something and are missing it in explorer.

[EmptySet]

I used Space Sniffer and found that there's ~7TB in the DeDup\Chunk Store. Is this normal behavior? I've used DeDup before but haven't seen this kind of an issue before.

[foggy]

Chunk store is a hidden folder, though I'm not sure how it takes part in disk space calculations. Here's good reading, btw.

[EmptySet]

So our backup volume is full (within the 3%) and preventing some backups from completing. I created a new repository on a separate drive, copied over a set of backups (say we have 5 jobs, I coped over 3 of the folders and all their contents) adjusted the backup jobs to point to the new repository and confirmed backups were running as expected. I have removed those backups from the original volume as necessary but it didn't free up the expected amount of space. I don't have dedup on at the OS level, so I'm assuming Veeam is running dedup. So my question is, even though I moved the backups off the original volume, does the dedup information still stay in the chunk store? Is there some Powershell to run that could reclaim unused space?

[foggy]

Did you use deduplication on that volume previously? Try to perform garbage collection.

[EmptySet]

I was hoping that would work... I tried it last night but there's an error that "the specified volume is not enabled for dedup." And doing Get-DedupVolume confirms it's not enabled. As far as I know it's never been enabled for this volume.

[foggy]

Then how the data could get to the chunk store?.. You can try to enable deduplication first. Anyway, this is out of the scope of Veeam B&R, so I'd ask Microsoft for assistance with this.

[EmptySet]

I don't disagree. I just wasn't sure if somehow Veeam used dedup that used the chunk store.

[foggy]

No, Veeam B&R inline deduplication is a completely different thing, it performs dedupe within it's backup files.

[EmptySet]

Thanks. Appreciate the help. I'll start working another angle.

[djwheele]

Hi All,
I am using Windows 2012 and Veeam Backup & Replication 9.

I have configured deduplication and it worked ok, but from some time I get the message about missing free space.
I have checked the volumes and I saw that I have 10.7 TB volume J:/

When I want to check it

Right Click on J:\ drive -> properties -> i see that there is Capacity 10,7 TB, used 10,6 TB free space 80GB

When I go to inside J:\ drive -> select all folders -> right click -> I See:

Size: 30,9 TB
Size on disk: 7,75 TB

My question is -> where is the rest of the free space? the difference between Windows reports on Computer (10,6 TB) and when You select all folders inside J:\ drive is about 3TB? How to recover this space?
What should I do?

If the size on disk is about 7,75 TB and it is stored on 10,7 TB drive, why there is written that only 80GB left?

Could anyone help me ?

[Mike Resseler]

Hi Marcin,

One of the things with dedupe is that files (or at least the chunks) are moved to a hidden folder (system volume information) and stored over there. You are not the first one that reports that the UI is not showing what it needs to show. But that is a Microsoft thing

Can you run this in PowerShell (as administrator): measure-dedupfilemetadata

That should give you the necessary information
Mike

[djwheele]

Thank You for Your answer.

I will run "your command" . Do You know how long it could take?
Is there any possibility to free more space ?

Marcin

[Mike Resseler]

It should only take a few seconds to give you the numbers.

I am not sure what you mean by "freeing more space"?

[djwheele]

so something is wrong on my server...
It is running about 2h and I still do not have a results ...

I have only something like this:

measure-dedupfilemetadata -path j:\
0/1 completed

<unknown>
Running, 00:00:00 remaining

Do You know what could happend ?
Marcin

[Mike Resseler]

Hi Marcin,
I gave you the wrong command. Sorry

Cancel that one and do this:
Get-DedupStatus -Volume J: (will give you free space, saved space, # of optimized files, # of in policy files)
Get-DedupVolume - Volume G: (will give you savedspace and savingsrate and the Usage Type)

The one you are doing indeed will take quite some time and the data known from that one can be interesting, but let it run on an evening and get the result in the morning

[djwheele]

I have a result right now

PS J:\> measure-dedupfilemetadata -path j:\

Path : {j:\}
Volume : J:
VolumeId : \\?\Volume{469c965e-86cf-4972-9e43-b28ae20c1927}
FilesCount : 281
OptimizedFilesCount : 54
Size : 26.76 TB
SizeOnDisk : 6.83 TB
DedupSize : 2.67 TB
DedupChunkCount : 73604189
DedupDistinctSize : 2.39 TB
DedupDistinctChunkCount : 69209945

[Mike Resseler]

It seems to me that you have quite a few files not optimized. What is your setting? If I am not mistaken every file not changed for the last 3 days will be optimized is the default.

[djwheele]

I have in my settings

"Deduplicate files older than 1 day"

In schedule

The process starts 2 times a day: 12 AM and 11 PM. Duration 12 hours

[Mike Resseler]

So either you have exclusions (a lot) or most of these files are being updated within the day. Can you verify?