Host-based backup of VMware vSphere VMs.
Post Reply
ASG
Enthusiast
Posts: 75
Liked: 5 times
Joined: Aug 08, 2018 10:19 am
Contact:

V12 - SureBackup starts with chkdsk

Post by ASG »

Hi,

Since moving to V12 oure surebackup Windows VMs start with chkdsk (which takes too long on some machines and stops the surebackup). I've since rebooted the source servers which don't start with chkdsk (so it seems there is no dirty bit set at the source) and waited for a new backup (even another full).

We're using surebackup quite a lot so this is only happening since V12.

Backupjobs are configured with "Enable Application-aware processing" enabled. So did I miss something when we switched to V12?

Thanks!
Gostev
Chief Product Officer
Posts: 31561
Liked: 6725 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: V12 - SureBackup starts with chkdsk

Post by Gostev »

Hi, no you did not miss anything: there were no related changes in V12 or anything you should have done after the upgrade. Thanks!
ASG
Enthusiast
Posts: 75
Liked: 5 times
Joined: Aug 08, 2018 10:19 am
Contact:

Re: V12 - SureBackup starts with chkdsk

Post by ASG »

Just a quick headsup - after creating a ticket with support (#05903648) it seems to be an NTFS thing. "There is a known bug generated by an incompatibility of the NTFS journaling's $LogFile between various versions." We're running Win2022 on the veeam Server.

However since it only started with V12 I've disabled the new "Disable Windows Firewall" Feature and the VMs won't chkdsk on boot in Surebackup anymore. So it's 'sort of' V12 since the disable Firewall is new :)
Gostev
Chief Product Officer
Posts: 31561
Liked: 6725 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: V12 - SureBackup starts with chkdsk

Post by Gostev »

Interesting. While what you mentioned is a new capability indeed, in general changing registry settings of the tested VM prior to booting it up is definitely not. SureBackup would not be possible if we did not do this for domain controllers going into application groups, for example.
ASG
Enthusiast
Posts: 75
Liked: 5 times
Joined: Aug 08, 2018 10:19 am
Contact:

Re: V12 - SureBackup starts with chkdsk

Post by ASG »

The other VMs that had the chkdsk problem were just plain VMs - no test scripts, no domain controllers etc... just "boot it up please" VMs. So while it seems it's not new the new feature is enabled by default for all VMs and mounts the disk where V11 wouldn't do it and would not trigger a chkdsk because of Server2022 mounting Server2012R2 NTFS.
YouGotServered
Service Provider
Posts: 171
Liked: 51 times
Joined: Mar 11, 2016 7:41 pm
Full Name: Cory Wallace
Contact:

Re: V12 - SureBackup starts with chkdsk

Post by YouGotServered »

Just curious - is this a Windows thing we are stuck with, or was there ever a workaround? Does this essentially trigger when:
1. Backup server is Windows Server 2022
2. Mounted server is 2012R2 with an NTFS filesystem and Veeam has to mount it to make some registry changes?

Sounds like this could end up affecting a good number of people as they migrate to newer Veeam versions, yet may have some old 2012 R2 Domain Controllers or something. Yes, I know that 2012R2 is going end of support soon, but that doesn't mean customers will upgrade, so it would be good to understand the scope of the potential issue :)

Thanks!
mkretzer
Veeam Legend
Posts: 1145
Liked: 388 times
Joined: Dec 17, 2015 7:17 am
Contact:

Re: V12 - SureBackup starts with chkdsk

Post by mkretzer » 1 person likes this post

We also have seen this with W2022 Mount Server mounting W2012R2 Domain Controller Disks. Seems like NTFS is behaving strange when mounting it from a much newer windows version. Mounting the same restore point to 2012R2 showed no error!
YouGotServered
Service Provider
Posts: 171
Liked: 51 times
Joined: Mar 11, 2016 7:41 pm
Full Name: Cory Wallace
Contact:

Re: V12 - SureBackup starts with chkdsk

Post by YouGotServered »

Thanks mkretzer. So my question now is:

1. Any workaround / registry hack Veeam can find or come up with on the MS side to help mitigate this? I know this is clearly an MS problem, but it will affect some Veeam customers, so if they could help find any sort of solution, that would be awesome.
2. How can I force mount a restore point on a particular server (for example, if my VBR backup server is Server 2022) to workaround this issue during a SureBackup? I can't recall if I've seen an option for that. Do I just need to tell the repository in Veeam to use a different non server 2022 mount server in the repo settings?

Thanks all - I know this issue is going to come up as we have some customers moving to Windows Server 2022 backup servers but they have some legacy apps / DC's still on 2012R2 and they will be until the cutoff date!
DerOest
Enthusiast
Posts: 71
Liked: 42 times
Joined: Oct 30, 2015 10:10 am
Contact:

Re: V12 - SureBackup starts with chkdsk

Post by DerOest » 1 person likes this post

We face this issue, too. Upgraded to v12 and did a migration from Server 2019 to a new Server 2022 with PostgreSQL.
-> SureBackup ran into chkdisk with Server 2012R2 VMs.

06007952 - Alexandru showed me this post

@YouGotServered: Changing the Mount Server in the Repository seems to be one workaround, yes. Our Veeam/VMware-Proxies are on 2019, so that would be a way, but this would affect all kind of things like instant recovery, too.
And for us that has two downsides, extra work (Firewall rules) and we'd lose access to the SSDs in our Veeam Hardware Server for the write-cache. We'd have to provide more capacity to the Proxy-VM.

So i guess we'll take the easy workaround (which is possible because we run very few SureBackup jobs with 2012R2 VMs): Increase timeouts in the Surebackup Job to simply let it run and finish the chkdisk.


(and I don't expect MS to fix that incompatibility in the short timeframe until we'll have those 2012R2 removed/migrated to 2022^^)
adriangalbincea
Novice
Posts: 3
Liked: 1 time
Joined: Jun 07, 2019 8:28 am
Full Name: Adrian Galbincea
Contact:

Re: V12 - SureBackup starts with chkdsk

Post by adriangalbincea » 1 person likes this post

ASG wrote: Mar 02, 2023 1:05 pm Just a quick headsup - after creating a ticket with support (#05903648) it seems to be an NTFS thing. "There is a known bug generated by an incompatibility of the NTFS journaling's $LogFile between various versions." We're running Win2022 on the veeam Server.

However since it only started with V12 I've disabled the new "Disable Windows Firewall" Feature and the VMs won't chkdsk on boot in Surebackup anymore. So it's 'sort of' V12 since the disable Firewall is new :)
Hi ASG, I can confirm that this was the issue for me too. Once this settings was turned off, the VMs booted right away without Checkdisk :)
Thanks for the tip. I was about to test it anyway but I said while I wait for the VM in troubleshooting mode, to google a bit about the issue.
NickKulkarni
Enthusiast
Posts: 29
Liked: 7 times
Joined: Feb 08, 2021 6:11 pm
Full Name: Nicholas Kulkarni
Contact:

Re: V12 - SureBackup starts with chkdsk

Post by NickKulkarni »

Hi All, I have not yet upgraded to V12 yet, I am still on V11 and I have this problem on a surebackup of our primary DC.
Never had it before but got it after I migrated the Veeam B & R Server to a new 2022 host. Curiously this is the only VM that is doing this all other surebackup jobs are fine.
I have rebooted the actual VM and it does not do a chkdsk so the VM has not got the dirty bit set.
Any ideas?
Cheers
Nick
tpayton
Veeam Software
Posts: 139
Liked: 90 times
Joined: Jan 16, 2013 5:27 pm
Full Name: Tyler Payton
Contact:

Re: V12 - SureBackup starts with chkdsk

Post by tpayton » 3 people like this post

The issue being discussed in this thread is addressed in https://www.veeam.com/kb4445
Tyler Payton
Customer Experience Manager / KB Manager
spiritie
Service Provider
Posts: 191
Liked: 40 times
Joined: Mar 01, 2016 10:16 am
Full Name: Gert
Location: Denmark
Contact:

Re: V12 - SureBackup starts with chkdsk

Post by spiritie »

Thanks god I found this forum post, this was driving me crazy :D
Redders
Lurker
Posts: 2
Liked: never
Joined: May 12, 2022 7:09 am
Full Name: l.redfern@crownoil.co.uk
Contact:

Re: V12 - SureBackup starts with chkdsk

Post by Redders »

Disabling the option to disable windows firewall as part of the lab has fixed this for me in the past.
Post Reply

Who is online

Users browsing this forum: Bing [Bot], Semrush [Bot] and 56 guests