Host-based backup of VMware vSphere VMs.
Post Reply
johnny@datafant.se
Enthusiast
Posts: 58
Liked: never
Joined: Jan 01, 2006 1:01 am

"Dirty shutdown" upon restore

Post by johnny@datafant.se »

Hello,

I am using Veeam Backup v2.0.1, VCB/SAN-mode and I have disabled the VMware VSS component in a newly installed Win2k3 O.S. and I am testing backup and restore to a renamed VM.
I have unchecked the Continue backup even if VSS fails-box.

The backup runs well.
The restore runs well.

But, the first time the new VM is powered on, Win2k3 complains that the previous shutdown was unsuccessful.

Does the VSS component fail anyway ?
This message should only appear if a crash-consistent backup has taken place, should it not ?

Regards,

Johnny Lundgren
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Post by Gostev »

Hello Johnny, this is normal, and is a nature of "hot" backups. Hot backups rely on snapshots, which are essentially crash-consistant images of your computer (same as if you would unplug power cord from your physical computer). Any modern OS and file system would automatically detect and recover from improper OS shutdown.

VSS is required to prevent loss of data by properly commiting all database and application transactions, flushing all application caches and buffers, as well as system cache, to the hard drive, and instructing any application requiring custom restore logic to perform the required steps on the next boot - before the actual snapshot is made (= power cord unplugged). But OS itself still remains running - it is not shutdown - this is in nature of "hot" backups. So, the snapshot is made after VSS processing but on running OS - resulting in transcationally consistent VM image from applications' or file system's "point of view", but from OS point of view it is still improper shutdown because OS never shuts down. The only way to achieve "proper" OS shutdown is to actually shutdown OS before backup (but that makes it "cold" backup).

Hope this helps!
br_sc
Novice
Posts: 7
Liked: never
Joined: Mar 23, 2010 11:27 am
Full Name: Michael Briegl
Contact:

[MERGED] Crash-consistent vs. Application-consistent

Post by br_sc »

I am having difficulties differentiating between a system start of a Windows VM that was restored to a "crash consistent" state and a system start of a Windows VM that was restored to an "application consistent" state.
Could anybody please clarify what behaviour to expect from a VM running "Windows Server 2003 R2" regarding the eventlog messages that should or should not show up in the Windows Event Log after a complete VM restore using Veeam Backup & Replication?
For example, In my Test-VM the eventlog entry (Source: Eventlog ID: 6008, "The previous system shutdown at ...was unexpected") shows up as the first entry on system startup after a complete VM restore, although the backup (apparently) was using "Veeam VSS" and should have created an "application-consistent" backup. Is this normal? Is this eventlog entry always showing up or is this a sure sign that something went wrong with the quiescing of the file system?
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: "Dirty shutdown" upon restore

Post by Gostev »

Veeam Backup leverages Windows VSS intergration to commit all application transactions ("application-level consistentcy") and file system operations ("file system level consistency"), so your system's backup will be in what we call "transactionally-consistent" state at the time when snapshot we use for backup is taken. OS will still report "dirty poweroff" upon restore though, because the functionality of detection of improper shutdown was not designed with virtualization and image-level backups in mind (see above for more). Thanks.
piloteight
Novice
Posts: 4
Liked: never
Joined: Mar 10, 2011 7:26 pm
Full Name: raymond wong
Contact:

[MERGED] rebooted without cleanly shutting down - Exchange 2

Post by piloteight »

HI,

Running Veeam backup 5.0.2.230. Veeam is running as a VM on windows 2008 r2. backed up exchange 2010 mailbox server. when I power it up, I got

"The system has rebooted without cleanly shutting down first. This error could be caused if the system stopped responding, crashed, or lost power unexpectedly.
"
I have the following checked:

1. virtual appliance
2. enable appliation-aware image processing
3. I left everything default

My understanding is that I will not get a crash state when I power up the backed up VM if I have "enable appliation-aware image processing" . Am I missing something ? Is there a " how to backup exchange 2010 using veeam" guide somewhere ?

Joe.
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: rebooted without cleanly shutting down - Exchange 2010

Post by tsightler »

This message is caused by the fact that the Veeam backup was taken while the system was still powered on and is not indicating that application-aware processing is not doing it's job. Basically, when Windows boots up, but doesn't see an equivalent "shutdown" it (it uses several methods to determine if there was a previous "shutdown") the system assumes that it was an unexpected shutdown and displays this message. So this message is technically correct, the system did not have a "clean shutdown" but this does not indicate that the system was not in a proper quiesced state via VSS.
piloteight
Novice
Posts: 4
Liked: never
Joined: Mar 10, 2011 7:26 pm
Full Name: raymond wong
Contact:

Re: "Dirty shutdown" upon restore

Post by piloteight »

Thank you for the reply. So how does one ensure that exchange 2010 was indeed completed a proper quiesced state via VSS ?


regards,

Joe.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: "Dirty shutdown" upon restore

Post by Gostev »

By enabling "application-aware processing" option in the job settings, and making sure that backup jobs completes successfully, without reporting any warnings or errors regarding guest processing or VSS.
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: "Dirty shutdown" upon restore

Post by tsightler »

The Windows application log will also have events that can indicate that everything is working as expected. For example, some of the events recorded on our Exchange 2010 server as as follows:

Code: Select all

Log Name:      Application
Source:        MSExchangeIS
Date:          7/4/2011 2:25:40 AM
Event ID:      9811
Task Category: Exchange VSS Writer
Description: Exchange VSS Writer (instance 18) has successfully prepared the database engine for a full or copy backup of database 'Public Folder Store'. 

Log Name:      Application
Source:        MSExchangeIS
Date:          7/4/2011 2:25:40 AM
Event ID:      9811
Description: Exchange VSS Writer (instance 18) has successfully prepared the database engine for a full or copy backup of database 'Mailbox Store'. 

Log Name:      Application
Source:        ESE
Date:          7/4/2011 2:25:40 AM
Event ID:      2005
Description: Information Store (5448) Shadow copy instance 18 starting. This will be a Full shadow copy. 

Log Name:      Application
Source:        MSExchangeIS
Date:          7/4/2011 2:25:40 AM
Event ID:      9608
Task Category: Exchange VSS Writer
Level:         Information
Description: Exchange VSS Writer (instance 0f426a87-7c4c-42a3-b4d1-79ef5b2d813d:18) has prepared for Snapshot successfully. 

Log Name:      Application
Source:        ESE
Date:          7/4/2011 2:25:41 AM
Event ID:      2001
Task Category: ShadowCopy
Level:         Information
Description:  Information Store (5448) Shadow copy instance 18 freeze started. 
Look for events like that in your log and you'll know it's all working if you don't trust what the Veeam console is reporting. You might also want to setup a SureBackup job to verify the recoverability of your Exchange environment.
Els M
Influencer
Posts: 10
Liked: never
Joined: Aug 17, 2009 7:40 am
Contact:

windows did not shut down succesffully started after restore

Post by Els M »

Hello All,
Thank you for your interest in my question.

I have taken a backup from a running windows virtual machine without any errors. Now I'm trying with 'import backup' to restore that machine on another ESXi hosts on another location. Restore goes well. At 1st boot: I got the black windows message stating that the windows machine did not shut down succesfully and I can choose to start in safe mode our start normally.
I don't suppose this is expected behaviour ? How can we avoid this ?
smartsys
Enthusiast
Posts: 32
Liked: 4 times
Joined: Sep 14, 2010 8:27 am
Full Name: Jeroen Leeflang
Contact:

Re: "Dirty shutdown" upon restore

Post by smartsys »

Hello Els M

As you can read above this is expected behaviour and you cannot avoid this.
It also isn't a problem. Just boot in Normal mode.
DanH
Novice
Posts: 4
Liked: 2 times
Joined: Mar 20, 2013 9:09 pm
Full Name: Maximus
Contact:

Re: "Dirty shutdown" upon restore

Post by DanH »

Hello Everyone,

Getting the "unexpected shutdown" after a restore is still the normal and expected behavior of hot backups correct?
I'm checking in case something changed for 2012 servers.

Thank you,
Dan
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: "Dirty shutdown" upon restore

Post by Vitaliy S. »

Hello Maximus,

Yes, nothing has changed.

Thank you.
ebeltran
Novice
Posts: 5
Liked: never
Joined: Jan 04, 2014 9:03 pm
Contact:

[MERGED] Restore behaves as if he had not turned off properl

Post by ebeltran »

Hi:
When a backup is recovered and the recovered virtual machine is started, the (Windows Server 2008 R2) operating system behaves as if the shutdown would not have been correct.
Is it normal when starting a VM recovered from a backup, it behaves as if he had not turned off properly ?.

a greeting
xkp
Novice
Posts: 6
Liked: never
Joined: Dec 03, 2016 6:08 pm
Contact:

[MERGED] "Windows did not shut down successfully" after rest

Post by xkp »

Hi all,
i m evalueting the Veeam Backup Essential (Veeam Backup and Replication 9.5) in a Simple Deployment environment (so, the Veeam Backup Server acts as 'control center', 'default backup proxy' and 'default backup repository'.

So i create a Backup job and in it I add two VM (Windows 7 Workstations) for the backup.
Both the VM are turned on when i execute the Backup Job.

As a test i decided to "delete from disk" of the ESXI one of the two machines and then perform a
"Entire WM Restore" of the deleted machine.
I add the VM from the backup job previously completed, select "Restore to the Original Location" and checked the "Restore VM Tags" button too.

The Restore job completed successfully but when i try to turn on the machine for the 1st time i can read the standar window message
"Windows did not shut down successfully. If this was due to the system not responding, or if the system was shut down.. etc etc"... asking me to "Boot it in Safe mode" or to "Start windows normally"

When i select the "Start Windows normally" the machine starts correctly and i have no problem, but still i m worried about the strange message and the fear the backup has been not properly done.

Any idea why i got this message? And is it a sign of a bad backup?
My ESXI is ESXI Essential 6.0.

thanks in advance
Cheers
Filippo
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: "Dirty shutdown" upon restore

Post by Vitaliy S. »

Filippo, this is absolutely normal situation when you're doing hot backups. The restored VM should not be affected by this. Please check out Gostev's post above for more info.
fsr
Enthusiast
Posts: 30
Liked: 1 time
Joined: Mar 27, 2019 5:28 pm
Full Name: Fernando Rapetti
Contact:

Re: "Dirty shutdown" upon restore

Post by fsr »

Hi. I suppose that Windows Server 2008 R2 is expected to show the same "Windows did not shut down succesfully" message after the VM is restored, right? Is there any version of Windows where this doesn't happen?

Regards
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: "Dirty shutdown" upon restore

Post by Vitaliy S. » 2 people like this post

Hi Fernando, I'm not aware of such versions.
Andreas Neufert
VP, Product Management
Posts: 6707
Liked: 1401 times
Joined: May 04, 2011 8:36 am
Full Name: Andreas Neufert
Location: Germany
Contact:

Re: "Dirty shutdown" upon restore

Post by Andreas Neufert »

Hi Fernando,

I have maybe an explanation.
If you Backup a Windows VM with Veeam guest processing it should boot directly into the OS and will show you a popup after login that asks you to document what happened with the server (detect that it was restored). This is total expected and is based on best practices.

VMware Tools quiescense do not have the enhanced application aware logic and I heard that it will boot and do not show this popup. This can be enabled by disabling the Veeam Guest processing and then enable the VMware tools quiescense within the Veeam Job Storage-Advanced - VMware section. This approach is not recommended as our own Guest processing is far more advanced.

There is one exception in this processing and this is when you restore a Windows Domain Controller that was backed up with our Guest processing.
If you restore this server with network connection enabled, we will automatically boot this server in Non-Authoritative restore mode. To do so we enable the "F8 Windows boot menue" and perform the steps automatically. If you touch the server in that processing we will stop the automatic process. For Active Directory Server restore you should only touch the console with mouse and keyboard when the server is ready for Windows User authentication to not interrupt the process.
fsr
Enthusiast
Posts: 30
Liked: 1 time
Joined: Mar 27, 2019 5:28 pm
Full Name: Fernando Rapetti
Contact:

Re: "Dirty shutdown" upon restore

Post by fsr »

Thanks, but i was referring to the following boot screen:

Image

I understand that the message appears, because the backup is made with Windows running. So, if your restore the VM, for Windows is like going from the instant of the backup directly to the boot process after the restore, so Windows incorrect shutdown detection gets fooled and thinks that the system was hard-rebooted, or something like that.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: "Dirty shutdown" upon restore

Post by veremin »

As Andreas's mentioned, you can enable Application-Aware image processing option to get rid to this message. Thanks!
fsr
Enthusiast
Posts: 30
Liked: 1 time
Joined: Mar 27, 2019 5:28 pm
Full Name: Fernando Rapetti
Contact:

Re: "Dirty shutdown" upon restore

Post by fsr »

But all the posts in the thread point to the screen been a normal, harmless consequence of hot backups, and one that is not possible to overcome. At least that's was what i understood from Vitaliy S. and Gostev's posts, specially this phrase seems difficult to misunderstand: "OS will still report "dirty poweroff" upon restore though, because the functionality of detection of improper shutdown was not designed with virtualization and image-level backups in mind (see above for more)."
I'm using Hyper-V and Application Aware is enabled and working in the jobs.
Andreas Neufert
VP, Product Management
Posts: 6707
Liked: 1401 times
Joined: May 04, 2011 8:36 am
Full Name: Andreas Neufert
Location: Germany
Contact:

Re: "Dirty shutdown" upon restore

Post by Andreas Neufert »

There are 2 things and it was not clear which one it is. Usually if you do Application aware processing you will see a popup after login that asks you why the server was restarted unexpectedly. This is normal and expected.

The message that you see is usually not visible if you do not have a domain controller that restart. With domain controller restore we set it in restore mode and will boot the server multiple times (please do not interrupt). I checked with the team and it seams that for Windows 7 this is as well expected by design of Win7.

I can not remember seeing this at Sever Operating Systems (non DC).
fsr
Enthusiast
Posts: 30
Liked: 1 time
Joined: Mar 27, 2019 5:28 pm
Full Name: Fernando Rapetti
Contact:

Re: "Dirty shutdown" upon restore

Post by fsr »

I'm seeing that black window in all Windows VMs. I always uncheck the option to start the VM automatically, as i want to be able to see the entire boot process and any messages on the screen. I tested it with a 2008 R2 DC, a 2008 R2 file server, a 2008 R2 database server, a 2008 with Exchange, and a Win 7 workstation. The virtualization hosts are also 2008 R2.
Could it be that the message is usually not visible, because it has a 30 sec countdown, so people normally doesn't see it?
Andreas Neufert
VP, Product Management
Posts: 6707
Liked: 1401 times
Joined: May 04, 2011 8:36 am
Full Name: Andreas Neufert
Location: Germany
Contact:

Re: "Dirty shutdown" upon restore

Post by Andreas Neufert » 1 person likes this post

I can check next week in my lab when I am back from vacation.
HannesK
Product Manager
Posts: 14287
Liked: 2877 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: "Dirty shutdown" upon restore

Post by HannesK » 1 person likes this post

@fsr: yes, that's normal because everything you test is the same Windows generation. Windows 7. I tested it with and without AAIP.

In later versions, the black screen does not appear anymore.
fsr
Enthusiast
Posts: 30
Liked: 1 time
Joined: Mar 27, 2019 5:28 pm
Full Name: Fernando Rapetti
Contact:

Re: "Dirty shutdown" upon restore

Post by fsr »

Excellent, thanks!
Post Reply

Who is online

Users browsing this forum: No registered users and 82 guests