Comprehensive data protection for all workloads
Post Reply
evander
Enthusiast
Posts: 86
Liked: 5 times
Joined: Nov 17, 2011 7:55 am
Contact:

Recovery procedure when Veeam server lost?

Post by evander »

Hi all - is there a documented recovery procedure for what steps to follow when you loose your Veeam Server?

Assume the following situation:

You have a full Virtual Environement with two sites, a Primary and a DR site

Veeam is an existing Virtual machine sitting on your primary site, running backups and replications locally and remotely to DR site. If the SAN at your primary site dies, what procedure should you follow to either recover your backups or your replicas? Bearing in mind your Veeam server was just hosed with the rest of your Primary Site servers.

So questions running through my head:

What is the best way to start the Replicas on DR site with no Veeam Server?
What is the best way to recover backups on DR site with no Veeam Server?
Assuming you re-install Veeam at DR site how do you import backups and replication jobs for recovery?

I am pretty sure this is a real world situation that I'm probably not the only one wondering about so I was really hoping there is an offcial documented Veeam procudure to follow?? ;-)
singy2002
Service Provider
Posts: 66
Liked: 1 time
Joined: Mar 22, 2010 8:43 am
Full Name: Andrew Singleton
Contact:

Re: Recovery procedure when Veeam server lost?

Post by singy2002 »

Hi,
In my case when we run through a full DR scenario we pretend that Veeam server has also gone.

The Replicas can just be powered on, (manually) although if you are only testing then a snapshot should be taken and fired up, this way once complete the snapshot can be reverted without needing to do another full replica.

The backups can be restored to a DR veeam box and the file(s) imported.

As for importing replica jobs, I am not aware of this being possible (sorry fi this is incorrect) as the replicas belong to the Veeam server they are created on.
evander
Enthusiast
Posts: 86
Liked: 5 times
Joined: Nov 17, 2011 7:55 am
Contact:

Re: Recovery procedure when Veeam server lost?

Post by evander »

Thanks Singy - I'm going to test that backup import function to make sure there are no unexpected issues that might arise so if ever the fan does get dirty its not too late already.

As a side note, I am also replicating my Veeam box (without Application Aware processing) which will, I assume allow me to get that box up first at the DR site and simply carry on from there as per normal, so technically I wouldnt need to import backups and replications jobs etc etc?
Ontrial
Influencer
Posts: 13
Liked: never
Joined: Dec 11, 2010 5:39 pm
Full Name: R
Contact:

Re: Recovery procedure when Veeam server lost?

Post by Ontrial »

an added complication is Vcentre.

If the replication to the DR site uses Vcentre in the primary site what do you do ? Bring up the Vcentre server dirty in the DR site ?

If the Vcentre server has its SQL on a SQL DB in the primary site what do you do? Bring up DB server dirty in the DR site ?

Messy!
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Recovery procedure when Veeam server lost?

Post by foggy »

Actually, there's a good topic on protecting the Veeam server. Please take a look, this approach allows you to quickly recover the Veeam server itself and start the required VMs using the Veeam native UI with the least effort.

Besides, there's always an ability to recover even without Veeam server in the DR site. As Andrew already mentioned, in v6, replicas are just common VM snapshots and can be started up using vSphere Client just fine (note, however, that in this case, re-IP rules will not be applied and you will not be able to perform advanced replica operations such as failback or permanent replica failover). For backups, there's a standalone utility (extract.exe) coming with every Veeam B&R installation that allows to restore VMs from any VBK file created by Veeam B&R, in case you do not have the product installed.
evander
Enthusiast
Posts: 86
Liked: 5 times
Joined: Nov 17, 2011 7:55 am
Contact:

Re: Recovery procedure when Veeam server lost?

Post by evander »

Thanks Foggy - gonna have a look at that thread right now...
Ontrial
Influencer
Posts: 13
Liked: never
Joined: Dec 11, 2010 5:39 pm
Full Name: R
Contact:

Re: Recovery procedure when Veeam server lost?

Post by Ontrial »

foggy wrote: Besides, there's always an ability to recover even without Veeam server in the DR site. As Andrew already mentioned, in v6, replicas are just common VM snapshots and can be started up using vSphere Client just fine.
This is messy because

1. If you want to test on a Saturday morning you have ruined your replicas that you brought up dirty, and therefore spend the following week cleaning up.
2. In a real DR you may have to go to version-1 replica .. how do you do you do that with the vsphere client ?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Recovery procedure when Veeam server lost?

Post by foggy »

1. What do you mean by saying ruined? Each restore point is a VM snapshot, and it will remain intact if you revert to it, as all disk changes from the running VM will be redirected into a separate snapshot file. And replication job also is not touched.

2. You can obviously revert to any of the stored restore points (any of the VM snapshots) with vSphere client.
ststrasser
Novice
Posts: 3
Liked: never
Joined: Mar 26, 2012 7:36 am
Contact:

What if main backup server goes down?

Post by ststrasser »

[merged]

I've been thinking: Our backup target is our main backup server as well... so if it goes down, how can I access and start my replicated machines, which are stored on our Synology NAS?

How do you all ensure you can access your backups?
Is it possible to manage backups from two seperate servers?
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Recovery procedure when Veeam server lost?

Post by dellock6 »

I usually install Veeam Server in a VM, and I create a replication job of the Veeam Server itself, so if the primary fails I can start the replica VM.
If I need to deal with FC or other "physical" elements, I add to the virtual Veeam Server a physical proxy/repository, and I use Veeam Server only as a central console.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
maxfontana
Influencer
Posts: 24
Liked: never
Joined: Nov 04, 2010 4:37 pm
Full Name: Massimiliano Fontana
Contact:

how to perform failover to replica if veeam crashes

Post by maxfontana »

[merged]

Hi there.
I have 2 hosts esxi 4.1 (A e B ) both with local storage and veeam installed as virtual machine on esxi B (vsphere essential) . Everyday it is backed up to a nas and in addition replicated to HOST A. Here comes the question:
if my host B dies (and with it my veeam vm), how do I start my replicated vms on host A ?
On the manual I've read that it is better to avoid starting a replicated vm directly from Vcenter and it would be better to use the veeam wizard .....this fact is confusing me as how can I use wizard if I have no available Veeam vm at all?
Many thanks in advance.
Max
Italy
Jfmoots
Veeam Software
Posts: 215
Liked: 26 times
Joined: Oct 28, 2011 3:26 pm
Full Name: James Moots
Location: Ohio, United States
Contact:

Re: how to perform failover to replica if veeam crashes

Post by Jfmoots » 2 people like this post

I always replicate my Veeam server to the DR side and I'd power that one on manually so that I could properly failover my other VMs.
JasonAF
Veeam ProPartner
Posts: 2
Liked: never
Joined: Dec 14, 2011 6:28 pm
Contact:

Replication architecture for single Veeam server?

Post by JasonAF »

[merged]

I need some input on v6 replication architecture...

I'm looking at a scenario where the basic need is to replicate from one building to another building on the same campus. They have a high-speed fiber link between the buildings, with vSphere hosts and FC SAN in each building. That being the case, my understanding from the user guide and other reading is that a single Veeam replication server would suffice, similar to an onsite replication scenario. I understand that the one server would act as both the source and target proxy, and thus would not perform any compression, which is fine since this is not over a slow WAN link.

However, the user guide also states "If possible, avoid powering on a replica manually as it may disrupt further replication operations or cause loss of important data. It is strongly recommended to use Veeam Backup & Replication functionality to perform failover operations."

So, first off, is it acceptable practice to use a single Veeam server in this scenario?

If so, should it reside at the source or target site? My first inclination would be to put it in the source site, but what if that site has a disaster? With there being no Veeam server at the target site in that scenario, my concern is about proper practice to perform a failover operation? I'd imagine the replica VMs in the target site could simply be powered on from the vSphere Client, but the warning above recommends against that.

What is best practice for this scenario? One server or two? If one, where should it reside and what is the acceptable/supported "failover" method to bring VMs in the target site online if source site is down?

Thanks
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Replication architecture for single Veeam server?

Post by dellock6 »

Hi Jason,
I usually design any replica operation, no matter is local, campus or wan, with at least two proxies, one at the source site and the other at the target site, and then I configure the replica jobs accordingly.
One "trick" for saving also the Veeam server is simply to replicate also this server (if it's installed on virtual...). In this way:
- if is have to failover any VM, you goes on using the original Veeam server for this operation
- if you have a complete site fail, you first power on Veeam server and from here you can go on managing all backup and replica jobs

If you are having a site failure, the least problem is loosing consistency in the replica operations, first goal is to power up all the failed VM...
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Replication architecture for single Veeam server?

Post by foggy »

dellock6 wrote:I usually design any replica operation, no matter is local, campus or wan, with at least two proxies, one at the source site and the other at the target site, and then I configure the replica jobs accordingly.
Another reason to have proxies in both locations is the ability to use hotadd mode both for reading and writing data (unless your single proxy has access to both datastores).

Jason, also please review this discussion above for manual failover considerations. Also, here is another good topic on this matter.
pcrebe
Enthusiast
Posts: 94
Liked: 1 time
Joined: Dec 06, 2010 10:41 pm
Full Name: CARLO
Contact:

Re: Recovery procedure when Veeam server lost?

Post by pcrebe »

Hi Jason,
I backup the veeam sql db at the end of every jobs with a post activity cmd so, if i need start the veeam_replica vm, i can restore the db at the nearest point in a few seconds. Naturally i put the db backup on the DR site :D

Carlo
applc
Influencer
Posts: 15
Liked: 2 times
Joined: Jun 01, 2010 3:48 pm
Full Name: Chris A
Contact:

Re: Recovery procedure when Veeam server lost?

Post by applc »

Ok if your VCenter and Veeam server are both located at the production site, how about this...(just thinking out loud).
1. Set up a second Veeam server at the DR site, just to control all replication jobs.
2. Replicate all VM's, including VCenter, from production site to DR site.
Then in a disaster, can you not just fire up the replicated VCenter, and then use the Veeam server at the DR site to control all the advanced B&R replication features?
All comments are welcome, as I haven't actually tried this. As I said, only thinking out loud.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Recovery procedure when Veeam server lost?

Post by Vitaliy S. »

Yes, this would also work!
RHDG
Lurker
Posts: 1
Liked: never
Joined: Jan 08, 2016 4:38 pm

Re: Recovery procedure when Veeam server lost?

Post by RHDG »

Interesting topic, but I'm missing something.

I'm only seeing scenario's where there is the need to go to the DR site.
An important part of disaster recovery planning is also how to roll-back to the primary site.
In worst case scenario, we'll need to configure a new (physical) Veeam B&R server when we roll-back to the primary site.

I suppose this would be possible with a Bare Metal Recovery (BMR)
I only found this blogpost about Veeam and BMR. Unfortunately, this only applies to Veeam Endpoint Backup.

So my question is: Whats the best practice of BMR for a Veeam B&R server? Is there a dedicated manual for this?

Thanks
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Recovery procedure when Veeam server lost?

Post by foggy »

Why BMR? There's a built-in Veeam B&R configuration backup functionality that allows you to fire up a fresh instance with all your previous jobs/settings on any Windows machine.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Recovery procedure when Veeam server lost?

Post by Vitaliy S. »

...and as to the BMR question, then you can definitely use Veeam Endpoint Backup for this purpose. There is no dedicated manual, cause this procedure is pretty straight forward:

1. Select backups you want to restore from via Recovery Media
2. Do the restore

P.S. here is a user guide for VEB talking about various ways to do that > http://helpcenter.veeam.com/endpoint/11 ... overy.html
OlafGuenther
Lurker
Posts: 1
Liked: never
Joined: Jun 01, 2016 5:30 pm
Full Name: Olaf Günther
Contact:

[MERGED] Backup and Restore SQL Server

Post by OlafGuenther »

Hello,

this is my first post and maybe a stupid question.

We have some VMs on Hyper-V Server running.
The Veeam Server runs also as VM.
Our SQL Server (with some Databases of our ERP Systems) runs also as VM.
The Veeam Database is on this SQL Server.

What if the SQL Server crashes completly?
How can I restore it? Is it possible to start Veeam without Database Access?

How looks the restore process?
Install a new SQL Server. Change Database Location to the new SQL Server. Import Veeam Seetings.
And start with Backup of the old SQL Server?

Thanks a lot!!
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

[MERGED] Re: Backup and Restore SQL Server

Post by PTide »

Hi,

It would be better if you kept Veeam database separated from other dbs.
Install a new SQL Server. Change Database Location to the new SQL Server. Import Veeam Seetings.
That will work, however an easier way would be to install Veeam Server anew and simply import the configuration. Please also see the thread for other users' experience.

Thanks
BibieR
Novice
Posts: 4
Liked: never
Joined: Mar 22, 2016 5:44 am
Full Name: Bibi Ramtu
Contact:

Re: Recovery procedure when Veeam server lost?

Post by BibieR »

Hi Guys,

we are currently having recovery issue. Veeam server went down and we lost all administrative tools and all programs wiped out. How can we restore VIB files stored on an external drive?
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Recovery procedure when Veeam server lost?

Post by Vitaliy S. »

You need to spin up a new backup server (Veeam), import backup files from the external drive and then do the restore.
Post Reply

Who is online

Users browsing this forum: No registered users and 68 guests