REFS performance issue "workaround"

[dellock6]

I believe the biggest benefit here is the monthly active full , which completely breaks the chain and restart a fresh one without involving too many deletions and transforms, which we have all understood by new are the most serious candidates to be the root problem. I don't see per-vm to be an issue to be honest.

[mkretzer]

We did an experiment: We have a volume where fast clone has got really slow. Fast clone for one specific job took 6 hours instead of ~25 minutes it took in the first few weeks. Now after an active full fast clone two days after the active full (it only had to merge 2 backup points!) took 4 1/2 hours. So in our case active full did not help at all!

[DaStivi]

hello
does anyone here having strange CPU issues with refs?
i noticed now a couple of times, that after a backup, when veeam starts to delete old vib files (perVM Backup) the backup repo server starts using all 8vCPUs and than locks up completly... no login is possible... right now i've restarted it a couple of times and after a few minutes the same behavoir started... i've removed the refs disks from my vm now and the server boots normally and i can login and right now i'm installing the latest windows cumulative updates...

btw. i'm on build 14393.1358 (and as far as i understand the "old" fixes should be in there allready: post247790.html#p247790)

i checked last syn full run of this backup repository, i sadly i've to notice the very long "fast clone" too, as it took 2h20min..

[kubimike]

@DaStivi same issue I had, contact Veeam open a case get the beta refs driver. In this case it fixed it for me.

[kubimike]

my fastclones are now taking 3 hours after fixing the retention issue. I am going through my backups before July 4th and it only took roughly an hour. Now it takes over 3 hours to make one.

[mkretzer]

The worst thing is; fast clones will continue to get slower and will never recover unless you reformat (or perhaps delete all the data). Thats why i had this idea for an workaround in this thread.

[jozne]

@DaStivi same issue I had, contact Veeam open a case get the beta refs driver. In this case it fixed it for me.

Hi , we are having the same issue. I made ReFS with 64k and stored some backups (4TB) to it for few days as incremental and now deleted all the backups through Veeam GUI
and the Server froze up with high CPU usage.

I try to ask if Support provides this beta driver.

[veeeammeupscotty]

@DaStivi same issue I had, contact Veeam open a case get the beta refs driver. In this case it fixed it for me.

Veeam is telling me there is no beta driver. Can you tell me your case number?

[mkretzer]

From what i understand the beta thing was some months ago and the driver got integrated in the latest updates - which did not help by the way!

[veeeammeupscotty]

Oh so you're referring to an actual Microsoft driver? Have you tried to install the hotfix mentioned in this thread which is supposed to be included in 9.5 U3? The symptoms there are different, high memory consumption, but the behavior makes me think it could be related. I have never seen the "TOO_MANY_REFERENCES" error in my environment, and the CPU peg issue has been sporadic. Each time I try something to fix it, it seems to go away for a while only to rear its ugly head again. It does seem to be triggered by synthetic full backups, which another user corroborates here.