Discussions specific to the Microsoft Hyper-V hypervisor
Post Reply
Poweruser
Expert
Posts: 189
Liked: 8 times
Joined: Jul 25, 2018 4:12 pm
Full Name: Poweruser
Contact:

Multiple VSS Consistency?

Post by Poweruser » Jul 31, 2018 12:17 am

Hi!
If i backup n hyper-v VMs which include: AD PDC, exchange, SQL server, etc.
How will veeam create the VSS/Snapshot?

are all VMs created in the same "second" of job execution or step by step?
exchange vm needs to be the same as AD PDC was in the same moment. is that always true?

and what about multiple VSS inside one machine? is this consistent too?
time(filesystem VSS) == time(sql VSS)?

Mike Resseler
Product Manager
Posts: 5749
Liked: 613 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Multiple VSS Consistency?

Post by Mike Resseler » Jul 31, 2018 5:49 am

Hi PowerUser.

Just to be clear, you have different VMs running, one AD server, one Exchange, one SQL server and they are all in the same job. Correct?

The way it exactly works is described here: https://helpcenter.veeam.com/docs/backu ... tml?ver=95

Multiple VSS inside one machine is always at the same time. That is not something that Veeam does, this is specific for the VSS subsystem by Windows. The moment we tell the VM to use VSS (we are the requestor) the freeze happens with all the VSS providers doing their job.

For different machines, the time in between can be different, but it should not be a problem at all. You can recover an AD and an Exchange server together even if there is a slight different in time of the VSS freeze. Is there a specific reason why they should be exactly the same?

Mike Resseler
Product Manager
Posts: 5749
Liked: 613 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Multiple VSS Consistency?

Post by Mike Resseler » Jul 31, 2018 5:49 am

Hi PowerUser.

Just to be clear, you have different VMs running, one AD server, one Exchange, one SQL server and they are all in the same job. Correct?

The way it exactly works is described here: https://helpcenter.veeam.com/docs/backu ... tml?ver=95

Multiple VSS inside one machine is always at the same time. That is not something that Veeam does, this is specific for the VSS subsystem by Windows. The moment we tell the VM to use VSS (we are the requestor) the freeze happens with all the VSS providers doing their job.

For different machines, the time in between can be different, but it should not be a problem at all. You can recover an AD and an Exchange server together even if there is a slight different in time of the VSS freeze. Is there a specific reason why they should be exactly the same?

Poweruser
Expert
Posts: 189
Liked: 8 times
Joined: Jul 25, 2018 4:12 pm
Full Name: Poweruser
Contact:

Re: Multiple VSS Consistency?

Post by Poweruser » Jul 31, 2018 10:22 pm

a) Okay, so its not consistent over multiple VMs. For Exchange/AD i think it can cause troubles in some cases, but for a slight difference it may work most times.

b) What about 2 VSS executions on the same VM? will it be sync oder slightlty async?
Reason is: Software useses Filesystem + SQL and these two need to be always consistent, means changes have to be tracked at the same moment. does VSS obey this? i fear not

c) The same as b), but inside one VSS execution on MSSQL. Will it freeze alle Databases at the same moment, then backup, then unfreeze all? If i check the logs, there are some seconds between each db freeze inside one instance. is it consistent or just per database only?

Mike Resseler
Product Manager
Posts: 5749
Liked: 613 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Multiple VSS Consistency?

Post by Mike Resseler » Aug 01, 2018 6:11 am

a) Actually, it should never be an issue at all. What might be an issue is that when you restore AD (let's say with a VSS snapshot of 22.00 hour) and an exchange (let's say with a VSS snapshot of 23.00 hour). Imagine that between 22.00 and 23.00 you created a new user in AD and gave him a mailbox. When you restore AD, obviously that user is not known, but it is known in Exchange. But even then, that mailbox will become 'orphaned' and when you recreate the user in AD, it will get itself fixed in Exchange.

b) No. Inside the VM the Microsoft subsystem takes care of the "job" and uses an internal mechanism so that every freeze is at the exact same moment

c) Yes, it will be frozen all at the same moment. I know that you can see differences in the log, but VSS does replay to make sure that the actual data presented in the VSS snapshot is at the exact same time. To explain it (hopefully, you can tell me if I failed ;-)) simply: The VSS requestor (Veeam in this case) requests to the VSS system of that VM to create a snapshot. The VSS system tells all the providers on that VM (SQL/ AD/ File/ Exchange/ ...) to prepare for backup. You will indeed see that they all prepare and that (logical actually) one provider might be done faster than the other. At that point in time, the VSS system uses its magic to get everything "in sync". Then it tells the requester (Veeam) that everything is ready and we grab that VSS snapshot. When we tell the VSS system that we are done, the system tells every provider to unfreeze (which again, will not be done at the exact same moment since some will be faster than others) Makes sense?

Poweruser
Expert
Posts: 189
Liked: 8 times
Joined: Jul 25, 2018 4:12 pm
Full Name: Poweruser
Contact:

Re: Multiple VSS Consistency?

Post by Poweruser » Aug 01, 2018 10:10 pm

Okay Mike, thanks for your detailed explaination.
As a big software vendor says: backup software needs to be able to do a "prepareToFreeze" command.
then this is what you described in c.

so if i use windows ntfs, shares, sql, ad, exchange, etc it should be all finally in sync per VM basis but not over all vms.
could it be possible that you add a sync over all vms? keep it frozen until everything is ready?
would be nice!

Mike Resseler
Product Manager
Posts: 5749
Liked: 613 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Multiple VSS Consistency?

Post by Mike Resseler » Aug 02, 2018 4:47 am

The problem with that is that I don't see it necessary (Don't forget that keeping that freeze state is very resource intensive so you want to keep it as limited as possible...)

Poweruser
Expert
Posts: 189
Liked: 8 times
Joined: Jul 25, 2018 4:12 pm
Full Name: Poweruser
Contact:

Re: Multiple VSS Consistency?

Post by Poweruser » Aug 02, 2018 12:24 pm

Resources and time is no matter for my backup jobs.
i got plenty of both :-)
iam thinking about adding a second job in the night for an offline backup.
shutting all vms down in the right order, do the backup, and restart them in the right order again.
i think that would be the safest and best way to make a backup.
additionally the online backup should run and clean up damn exchange logs. ;-)

anyway i think it is nice if the state is equal over all vms.
its the safest way because everything belongs together as one "instance" in my installation.

Mike Resseler
Product Manager
Posts: 5749
Liked: 613 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Multiple VSS Consistency?

Post by Mike Resseler » Aug 02, 2018 5:48 pm

Obviously that is a safe way but as said, it is not really necessary. But being for 20 years in the backup world, I understand perfectly your concerns

PS: Make sure you test surebackup to see if different VM's can start in a quarantined environment and if they work well. You will love this functionality, especially since you have enough resources ;-)

Poweruser
Expert
Posts: 189
Liked: 8 times
Joined: Jul 25, 2018 4:12 pm
Full Name: Poweruser
Contact:

Re: Multiple VSS Consistency?

Post by Poweruser » Aug 04, 2018 11:54 am

Yes i think i will love this functionality, too. But i dont know how to start and will have to check it soon, when i find time. Seems to be more than a 5 minute job to set it up the first time.
The big software vendor which uses sql+filedatabases says you SHOULD have offline backup in every case next to online backups because it is safe in this case only.
They fear about differences between smb share and sql state because they both are linked together and the snapshot needs to be at identical time for filesystem+sql.

Mike Resseler
Product Manager
Posts: 5749
Liked: 613 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Multiple VSS Consistency?

Post by Mike Resseler » Aug 06, 2018 6:21 am

I am not going to talk about that software vendor or try to understand why they state that. What I assume is that the box is using SQL server and the data of the databases are stored on that SMB share. So I still stand with the fact that it doesn't need to be the identical time but if you make huge changes to the config on the server in between the backup of the data and server, you might run into issues

Poweruser
Expert
Posts: 189
Liked: 8 times
Joined: Jul 25, 2018 4:12 pm
Full Name: Poweruser
Contact:

Re: Multiple VSS Consistency?

Post by Poweruser » Aug 06, 2018 10:03 pm

they do it a bit tricky.
first all mssql databases are saved onto the shared. i dont know why.
second they have an own database which is something like sqlite or more uglier. its sits there and contains metadata, usersettings in ini files and thousands other legacy things like logs, configs, etc they have developed in the last centuries from dos to windows.
means: the main stuff is sql but all the older stuff and the management stuff goes onto smb.

edit: nevertheless i tried to to an offline backup as suggested with prefreeze jobs.
but it did not work, because veeam has an fixed script timeout.
their offline backup prejob script is taking much time.

1. database check
2. online backup to folder
3. shutdown mssql db
4. do a loop ping for 15 minutes to wait that sql is safe down.
lets guess after 20-30 minutes or later veeam will get its script done.
but veeam does not want to wait as long as desired :-(

Mike Resseler
Product Manager
Posts: 5749
Liked: 613 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Multiple VSS Consistency?

Post by Mike Resseler » Aug 07, 2018 5:24 am

I probably don't want to know what the name of that solution is... How long before a database can shutdown?

I believe you can change the timeout of the pre-script by looking for this key in the registry:

HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication" PreJobScriptTimeoutSec. If I am not mistaken the default is 900 seconds. So you can change that to a bigger number.

Poweruser
Expert
Posts: 189
Liked: 8 times
Joined: Jul 25, 2018 4:12 pm
Full Name: Poweruser
Contact:

Re: Multiple VSS Consistency?

Post by Poweruser » Aug 07, 2018 8:09 am

as in other threads i changed these values and the more specific vss timeout.
but veam timeouts after 10 minutes.

Mike Resseler
Product Manager
Posts: 5749
Liked: 613 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Multiple VSS Consistency?

Post by Mike Resseler » Aug 07, 2018 8:12 am

Did you restart the service?

Poweruser
Expert
Posts: 189
Liked: 8 times
Joined: Jul 25, 2018 4:12 pm
Full Name: Poweruser
Contact:

Re: Multiple VSS Consistency?

Post by Poweruser » Aug 07, 2018 10:19 pm

yes, the main backup service.
but the kb article says default is another value i think 15m instead of 10.
maybe the article is outdated for vss timeout.
btw there is another similiar topic in this board about this.

Mike Resseler
Product Manager
Posts: 5749
Liked: 613 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Multiple VSS Consistency?

Post by Mike Resseler » Aug 08, 2018 6:21 am

In that case I would advice you to open a support call. To the best of my knowledge, this setting should work and should timeout only after the time in the setting

Poweruser
Expert
Posts: 189
Liked: 8 times
Joined: Jul 25, 2018 4:12 pm
Full Name: Poweruser
Contact:

Re: Multiple VSS Consistency?

Post by Poweruser » Aug 08, 2018 10:40 pm

about the timeout part: it was resolved in thread: (i hope)
https://forums.veeam.com/veeam-backup-r ... ml#p290113

Post Reply

Who is online

Users browsing this forum: No registered users and 7 guests