Corrupted Files on Win2016 Deduplication

Availability for the Always-On Enterprise

Re: Corrupted Files on Win2016 Deduplication

Veeam Logoby JonSD » Tue Apr 25, 2017 11:46 am

Do your scrubs run on Sunday RGijsen?
JonSD
Novice
 
Posts: 5
Liked: never
Joined: Tue Aug 05, 2014 2:11 pm
Location: San Diego, CA
Full Name: Jon Bucud

Re: Corrupted Files on Win2016 Deduplication

Veeam Logoby RGijsen » Tue Apr 25, 2017 12:03 pm

Actually I initiate the dedupe myself by running a script after the last backup job, to prevent scheduled dedupe jobs to run when backups are still being put on that volume. I run this script:

Start-DedupJob -Volume V: -Type Optimization -memory 70
if ((get-date).DayOfWeek -eq 'Thursday') {Start-DedupJob V: -Type GarbageCollection}
if ((get-date).DayOfWeek -eq 'Friday') {Start-DedupJob V: -Type GarbageCollection}
if ((get-date).DayOfWeek -eq 'Saturday') {Start-DedupJob V: -Type GarbageCollection}
if ((get-date).DayOfWeek -eq 'Sunday') {Start-DedupJob V: -Type GarbageCollection}

I run the garbagecollection on days when Active Fulls run in order to dedupe most of the full backup files again. We're quite short on storage at the moment, which is why I do that.
RGijsen
Enthusiast
 
Posts: 55
Liked: 4 times
Joined: Fri Oct 10, 2014 2:06 pm

Re: Corrupted Files on Win2016 Deduplication

Veeam Logoby Delo123 » Mon May 01, 2017 7:50 am

Guys, just to be sure. You all have set Dedupe to General purpose file server or virtual backup server and not to VDI Server? VDI Servers already caused tons of issues in 2012...
For us in 2016 with all updates and Virtual Backup Server 0 issues until now with over 140TB of Data on 3 Volumes.
Delo123
Expert
 
Posts: 348
Liked: 94 times
Joined: Fri Dec 28, 2012 5:20 pm
Full Name: Guido Meijers

Re: Corrupted Files on Win2016 Deduplication

Veeam Logoby RGijsen » Mon May 01, 2017 7:52 am

We use general purpose.
RGijsen
Enthusiast
 
Posts: 55
Liked: 4 times
Joined: Fri Oct 10, 2014 2:06 pm

Re: Corrupted Files on Win2016 Deduplication

Veeam Logoby Delo123 » Mon May 01, 2017 8:06 am

Yeah, i just noticed in the other thread.
Did you enable OptimizeInUseFiles again? Because this is the actual difference to VDI if i'm not mistaken. At least with OptimizeInUseFiles we ran into a lot of issues years ago.
Delo123
Expert
 
Posts: 348
Liked: 94 times
Joined: Fri Dec 28, 2012 5:20 pm
Full Name: Guido Meijers

Re: Corrupted Files on Win2016 Deduplication

Veeam Logoby RGijsen » Mon May 01, 2017 8:54 am

It's enabled now but I've had it disabled if I remember correctly to no avail. But I'll disable it right now, and see how that goes.

[edit]
no, I'll change the setting tomorrow. I'm curious if it strikes us again on Tuesday as seems to be more rule than exception in the past.
RGijsen
Enthusiast
 
Posts: 55
Liked: 4 times
Joined: Fri Oct 10, 2014 2:06 pm

Re: Corrupted Files on Win2016 Deduplication

Veeam Logoby Delo123 » Mon May 01, 2017 10:55 am

Ah ok, as said. Optimize files in use failed on us every time the dedupe process was still running (big increments and fulls) as soon Veeam tried to access the files.
Anyway, it doesn't make sense to process in use files anyway (very scary!).

Ok, let's see what happens afer tomorrow then
Delo123
Expert
 
Posts: 348
Liked: 94 times
Joined: Fri Dec 28, 2012 5:20 pm
Full Name: Guido Meijers

Re: Corrupted Files on Win2016 Deduplication

Veeam Logoby RGijsen » Tue May 02, 2017 8:20 am 1 person likes this post

Sure, scary, but I trigger the dedup myself after all jobs that write are done. However, during the copy jobs to offsite some files are still accessed for read ofcourse, which could very well be deduped at the same time in my oppinion. Still I've disabled it now. I've not had the error last night by the way breaking a streak of several weeks of error on tuesdays.
RGijsen
Enthusiast
 
Posts: 55
Liked: 4 times
Joined: Fri Oct 10, 2014 2:06 pm

Re: Corrupted Files on Win2016 Deduplication

Veeam Logoby Delo123 » Tue May 02, 2017 8:59 am

But you had it enabled still yesterday right? So the reason for the job not failing is still unknown? :(
I know what you mean regarding reading only, but actually i do not see a single reason why one should want to actually dedupe in use files (regarding to backups) so yeah we disabled it on every repository just in case...

Ok, let's see how it goes for you now... How much data do you have on your Dedupe Volumes anyway? Volumes are around 62TB also in your case?
Delo123
Expert
 
Posts: 348
Liked: 94 times
Joined: Fri Dec 28, 2012 5:20 pm
Full Name: Guido Meijers

Re: Corrupted Files on Win2016 Deduplication

Veeam Logoby Delo123 » Mon May 08, 2017 10:40 am

RGijssen. How are things after disabling the open files thing? Or something new popped up?
Delo123
Expert
 
Posts: 348
Liked: 94 times
Joined: Fri Dec 28, 2012 5:20 pm
Full Name: Guido Meijers

Re: Corrupted Files on Win2016 Deduplication

Veeam Logoby Ctek » Mon May 08, 2017 2:12 pm

I confirm the same deduplication behavior described in this thread, with a fully patched W2016 server. All our VBK files are <500GB.

:(

This is not ready for prime time yet IMHO.
VMCE 9 Certified - Systems Administrator
Ctek
Service Provider
 
Posts: 38
Liked: 3 times
Joined: Wed Nov 11, 2015 3:50 pm
Location: Canada

Re: Corrupted Files on Win2016 Deduplication

Veeam Logoby Delo123 » Mon May 08, 2017 2:15 pm

Ctek: Do you by any change have OptimizeInUseFile enabled or set Data deduplication type to "Virtual Desktop Infrastructure (VDI) server"?
Delo123
Expert
 
Posts: 348
Liked: 94 times
Joined: Fri Dec 28, 2012 5:20 pm
Full Name: Guido Meijers

Re: Corrupted Files on Win2016 Deduplication

Veeam Logoby Ctek » Mon May 08, 2017 3:52 pm

OptimizeInUseFile is enabled (Probably by default, I have not changed this setting on my own).

Data deduplication type is set to Virtualized backup server.
VMCE 9 Certified - Systems Administrator
Ctek
Service Provider
 
Posts: 38
Liked: 3 times
Joined: Wed Nov 11, 2015 3:50 pm
Location: Canada

Re: Corrupted Files on Win2016 Deduplication

Veeam Logoby Delo123 » Mon May 08, 2017 5:50 pm

My bad... We have set is for "general file server", this disables OptimitzeinUseFiles, but at least OptimizePartialfiles should be disabled (which is the VDI setting). Then again it might be safer to go for general file server, but just guessing :(
Delo123
Expert
 
Posts: 348
Liked: 94 times
Joined: Fri Dec 28, 2012 5:20 pm
Full Name: Guido Meijers

Re: Corrupted Files on Win2016 Deduplication

Veeam Logoby RGijsen » Tue May 09, 2017 6:17 am

Delo123 wrote:RGijssen. How are things after disabling the open files thing? Or something new popped up?

Last night I hit the issue again with OptimizeInUseFiles disabled. And again on Tuesday. Again on the same backup job (not even the biggest one).
RGijsen
Enthusiast
 
Posts: 55
Liked: 4 times
Joined: Fri Oct 10, 2014 2:06 pm

PreviousNext

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: bstreza, Google [Bot], iColin and 36 guests