Slow backup file merge with REFS

Availability for the Always-On Enterprise

Re: Slow backup file merge with REFS

Veeam Logoby mkretzer » Sat Jun 03, 2017 11:11 pm

One more thing: Our write speed is extemly slow again just because two synthetics (fast clone) are running. Disk latency is still between 2 and 4 ms so disk is not loaded. Before the synthetics started speed was normal.
mkretzer
Expert
 
Posts: 341
Liked: 74 times
Joined: Thu Dec 17, 2015 7:17 am

Re: Slow backup file merge with REFS

Veeam Logoby suprnova » Sun Jun 04, 2017 2:31 pm

That's really strange, we've had basically all of the other issues, but our write speeds are normal while fast clone synthetics are running. We are running the experimental refs.sys driver though so maybe that is a potential fix. But Veeam support is unable to report what exactly the new driver fixes.
suprnova
Service Provider
 
Posts: 21
Liked: never
Joined: Fri Apr 08, 2016 5:15 pm

Re: Slow backup file merge with REFS

Veeam Logoby mkretzer » Sun Jun 04, 2017 2:53 pm

I mean it must be something in the REFS driver. With NTFS there is REAL LOAD on the volume while merging but we never ever had any issues getting the data to disk while all the merges ran. We basically never had to care if the active fulls were going at the same time as synthetics - it just worked (same backend storage).

Where do i get that REFS driver and is it usable for production?
mkretzer
Expert
 
Posts: 341
Liked: 74 times
Joined: Thu Dec 17, 2015 7:17 am

Re: Slow backup file merge with REFS

Veeam Logoby suprnova » Sun Jun 04, 2017 10:42 pm

Our issue has always been merges with NTFS, just way too slow, but we are also a service provider where there may be 100+ merges overnight. Veeam support can provide the driver. It's tough to say if it is production worthy or not, but I can say it has not made things worse.
suprnova
Service Provider
 
Posts: 21
Liked: never
Joined: Fri Apr 08, 2016 5:15 pm

Re: Slow backup file merge with REFS

Veeam Logoby mkretzer » Thu Jun 08, 2017 5:32 am

Ok setting all the REFS Reg settings Microsoft recommends now leads to even slower merges and also slower write speed to the paralell running backups. Memory usage is lower but now our backup window is too long... We will remove the settings again and increase retention as a test.
mkretzer
Expert
 
Posts: 341
Liked: 74 times
Joined: Thu Dec 17, 2015 7:17 am

Re: Slow backup file merge with REFS

Veeam Logoby mkretzer » Sun Jun 11, 2017 7:11 am

Without deleting retention points our merges doubled in speed! We are not at the original "fast clone" speed from the beginning but its much better. Even an active full does not immediately stall when a merge is running in parallel.

Right now we have enough space to host even another week so we will see if the merges will even get faster next weekend.

I wonder: is there anything i can optimize with better hardware for the repo server? I simply cannot see which ressource is limiting the REFS to merge faster/delete restore points without issue. RAM is no issue anymore since we increased to 384 GB, CPU usage is low and disk queue of the REPO is 0. Buying a new server would still be cheaper than buying more storage and going back to NTFS :-)
mkretzer
Expert
 
Posts: 341
Liked: 74 times
Joined: Thu Dec 17, 2015 7:17 am

Re: Slow backup file merge with REFS

Veeam Logoby mkretzer » Sun Jun 18, 2017 9:11 am

Ok even without deletion of restore points everything got even slower with 6 weeks of backups on disk. We will go back to NTFS as we see no way to get this working right now...
mkretzer
Expert
 
Posts: 341
Liked: 74 times
Joined: Thu Dec 17, 2015 7:17 am

Re: Slow backup file merge with REFS

Veeam Logoby mikegodwin » Mon Jul 24, 2017 4:50 pm

Has anyone tried disabling FileIntegrity in ReFS to see if it improves performance?
mikegodwin
Enthusiast
 
Posts: 52
Liked: 1 time
Joined: Fri Oct 12, 2012 12:28 am
Full Name: Mike Godwin

Previous

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: ayecee, Bing [Bot], Google [Bot] and 10 guests