Comprehensive data protection for all workloads
foggy
Veeam Software
Posts: 20415
Liked: 1944 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Check Disk During Boot

Post by foggy »

Does the corresponding backup job have AAIP enabled?

JoshuaPostSAMC
Expert
Posts: 124
Liked: 21 times
Joined: Jul 30, 2015 7:32 pm
Contact:

Re: Check Disk During Boot

Post by JoshuaPostSAMC »

Yes, this particular VM is in a job with AAIP enabled, although this server does not have SQL running on it.

All of my proxies that this job uses are set to Automatic Transport Mode. This VM did use Hotadd last night, but nearly all of my VM do except for my Exchange servers which are set to use a proxy that is NBD.

foggy
Veeam Software
Posts: 20415
Liked: 1944 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Check Disk During Boot

Post by foggy »

Do you see similar behavior on the original VM, if you try to reboot it?

JoshuaPostSAMC
Expert
Posts: 124
Liked: 21 times
Joined: Jul 30, 2015 7:32 pm
Contact:

Re: Check Disk During Boot

Post by JoshuaPostSAMC »

In previous instances, no. The original VM reboot successfully. At this point I went ahead and scheduled a chkdsk on next reboot of the original VM to be safe so until it does a reboot and runs that I won't have an accurate test in SureBackup

foggy
Veeam Software
Posts: 20415
Liked: 1944 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Check Disk During Boot

Post by foggy »

Don't forget to take a fresh backup after that.

mcz
Veeam Legend
Posts: 512
Liked: 87 times
Joined: Jul 19, 2016 8:39 am
Full Name: Michael
Contact:

[MERGED] DC is running checkdisk in SureBackup Lab

Post by mcz »

Hi everybody,

my SureBackup jobs are failing the last days and the reason is obvious: My DC is running a checkdisk during startup and therefore the job runs into the timeout. Now you could easily just increase the time in the settings but that's not the question. The question is why does it run checkdisk?? When I restart my DC in the production, no checkdisk runs there - so the filesystem should be ok. The backup itself has healthchecks after every SureBackup run and once a week - I assume the backup is fine.

So something with the filesystem on the vm in the lab seems to be wrong - but why could it ever happen?

Thanks in advance!

HannesK
Veeam Software
Posts: 8866
Liked: 1593 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: DC is running checkdisk in SureBackup Lab

Post by HannesK »

Hello,
application aware backup creates a VSS snapshot. I guess it has something to do with that. I also guess that when you rebooted your production DC, that you did not do authoritative / non-authoritative restore (which happens when you select the roles in SureBackup).

For definite answer, please contact support.

Best regards,
Hannes

mcz
Veeam Legend
Posts: 512
Liked: 87 times
Joined: Jul 19, 2016 8:39 am
Full Name: Michael
Contact:

Re: DC is running checkdisk in SureBackup Lab

Post by mcz »

Hi Hannes,

investigation is still going on, case # 04326276. When I do a Instand VM recovery or even start a replica manually, everything is fine. As soon as SureBackup starts that vm, checkdisk is run. Looks like the DC preparation is doing something which breaks the filesystem.

foggy
Veeam Software
Posts: 20415
Liked: 1944 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: CHKDSK During SureBackup Boot

Post by foggy »

Hi Michael, I've merged your thread into an existing one discussing similar issues, please review as it contains a few hints that might be helpful. Thanks!

mcz
Veeam Legend
Posts: 512
Liked: 87 times
Joined: Jul 19, 2016 8:39 am
Full Name: Michael
Contact:

Re: CHKDSK During SureBackup Boot

Post by mcz »

Hi Alexander,

thanks for the merge. Changing transport mode to nbd (from hotadd) hasn't helped. Source filesystem is clean. There's nothing I could do right now, so I'll wait for the response from support.

foggy
Veeam Software
Posts: 20415
Liked: 1944 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: CHKDSK During SureBackup Boot

Post by foggy »

Since it has started out of nowhere - does AAIP complete successfully during the last backups?

mcz
Veeam Legend
Posts: 512
Liked: 87 times
Joined: Jul 19, 2016 8:39 am
Full Name: Michael
Contact:

Re: CHKDSK During SureBackup Boot

Post by mcz »

yeah, sure. That was one of the first things I've checked. It doesn't make sense to me at all why it came all of a sudden.

nitramd
Veeam Legend
Posts: 296
Liked: 81 times
Joined: Feb 16, 2017 8:05 pm
Contact:

Re: CHKDSK During SureBackup Boot

Post by nitramd »

Michael,

It sounds like the disk's dirty bit has been set for some reason; it's usually set when a disk is improperly shut down.

Follow this link for strategies: https://superuser.com/questions/469521/ ... every-boot

Maybe experimentation and testing are in order.

Hope this helps and good luck.

mcz
Veeam Legend
Posts: 512
Liked: 87 times
Joined: Jul 19, 2016 8:39 am
Full Name: Michael
Contact:

Re: CHKDSK During SureBackup Boot

Post by mcz »

Hi foggy,

case # 04326276. Looks like since Windows 10 2004 (and also in the latest october release) the ntfs driver is corrupting the disk. Since veeam has a direct link to microsoft, I hope that there's a conversation going on??

Gostev
SVP, Product Management
Posts: 28957
Liked: 5303 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: CHKDSK During SureBackup Boot

Post by Gostev »

The direct link we (R&D) have is to the ReFS dev team only. But normally, our support opens cases with the Microsoft Premier Support automatically as soon as the issue is confirmed to be on their side. It's actually very common for our support to open cases with other vendors because so many issues our users run into have nothing to deal with Veeam. And we in R&D simply don't track all these cases, but the engineer on your case will be able to tell you if they already opened a case.

mcz
Veeam Legend
Posts: 512
Liked: 87 times
Joined: Jul 19, 2016 8:39 am
Full Name: Michael
Contact:

Re: CHKDSK During SureBackup Boot

Post by mcz »

Thanks Anton. The engineer told me that he's waiting for a response from T3 and I have already asked if they talked to Microsoft. Maybe they did but at least it wasn't confirmed to me.

mcz
Veeam Legend
Posts: 512
Liked: 87 times
Joined: Jul 19, 2016 8:39 am
Full Name: Michael
Contact:

Re: CHKDSK During SureBackup Boot

Post by mcz » 1 person likes this post

Update: My engineer wrote me the following:
The MS case was opened. Title: "NTFS corruption when mounting a disk to a different Windows version"
So finally the case has reached Microsoft. I'm still surprised that they didn't fix it (or maybe even didn't recognize it???) because the latest Windows 10 version has the same bug, so it exists since may...

The ugly part here is that you're limited when you're running such an affected version. Engineer told me the following:
The data is not safe only during SureBackup DC configuration, or replica Re-Ip (in that case Veeam is mounting replicated VM hard disk to the proxying server), or Windows FLR. In those cases, it would be better to avoid assign Veeam server or proxy server role to the Windows 10 2004 servers. In all other cases everything should be OK (Backup, restore, replica, other OS FLR,etc... ).
I'm sure I don't know every detail of this case (how should I) but it looks like this NTFS bug is quite severe for a properly working installation and so it still surprises me that it takes so long overall...

Post Reply

Who is online

Users browsing this forum: colmennis, Google [Bot] and 73 guests