Inability to restore multiple VMs at once

[TroyB]

Can you please look into adding this feature in the near future? As much as I love your product, we will have to continue to use a competing product until this is an option.

Please let me know when it's implimented and I'll review your product again.

[Gostev]

Restoring multiple VMs at once was possible since version 1.0, what do you mean?

[TroyB]

I mean that when I go through the restore steps, I can't select multiple VMs at one time. I called your support and verified that this isn't an option. If you're telling me otherwise, please explain how to do it.

[Gostev]

I mean that you can start multiple restore jobs one by one by going through the restore wizard a few times. The wizard is just a few steps, plus each VM is likely to be restored to different host/datastore anyway (cannot put everything on one datastore/host), so bulk restores with the same settings like you are describing are sometimes not even possible.

I agree restore jobs may come helpful in certain scenarios, but it was not very common request before.

[TroyB]

You are incorrect. I will not say the competing product but I AM able to restore multiple VMs at once there. I simply select all the VMs I need to restore and it then pulls up the list and allows me to select the datastores / hosts for each one.

In a DR situation the time saved would be HUGE. If you fail to see that and don't want to take my feature request into consideration that's perfectly fine. I will continue to use my current product then.

[Gostev]

We take any request into consideration, but I need to have good understanding of each request, especially when talking about some minor missing features which absence have not been a showstopping problem for 10 thousands plus of other customers.

So if you can make some clarifications, it would help greatly. Specifically, what I don't understand is where "HUGE" time savings come from, if you still have to select restore parameters for each VM, with one approach or another. To me, it sounds like we are talking about difference in just a few extra mouse clicks to launch the wizard for every new VM. I don't see how this can become a showstopper. Given how rarely you need to restore more than 1 VM, and how long restore of even 10 VMs would take (many hours); the time it would take to configure and start separate restore jobs is negligible and cannot really be showstopper. Do you agree with that? If not, do you have some ammo that would help me bump the priority of this feature when talking to my management? We have hundreds of other pending nice-to-have features, so we have to prioritize them by real value they bring to the product.

Don't get me wrong, as I've already said above, this feature is nice to have. However, my understanding and use cases behind this feature was totally different:
- Ability to schedule restore jobs instead of running immediately
- Better tracking for restores (to have restore sessions history, similar to backup jobs)
- Bulk restores for datacenter migrations

I would understand if the lack of these features would become showstopper for someone, but this is not what you are asking for...

[TroyB]

Let's use my environment as an example:

- 2 seperate sites
- 5 hosts per site
- Approx. 300 VMs total (So approx. 150 per site)
- We keep enough resources available on each sites hosts to house twice as many VMs

Now, the way we currently do backups, is nightly we backup every VM from site to site on seperate SAN storage.

So, in a DR situation and 1 site were to go down, I would need to restore approx. 150 VMs onto the hosts at the site that's still up.

In the other product, to do this I would select all the priority one VMs all at once, pull up the list, selct the hosts/datastores and let it do it's job while I'm freed up to work on other abviously pressing matters since I just lost an entire site.

In you're product I'm starting each and every restore job one at a time having to go through a wizard that adds extra seconds for each and every VM.

Yes, in a DR situation, those seconds per VM wouldl be quite "HUGE" and valuable to me. I'm sorry you don't seem to place the same value on my time as I do.

[Gostev]

If you are already keeping spare resources in each site to house twice as many VMs, logical choice would be to use replication capabilities our product provides. Just replicate your VMs between the 2 sites, and this way you will have them ready to go in case you lose one of the sites - all you will need to do is power them on, pretty much instant recovery! What do you think?

Restoring 150 VMs seems quite a task in all cases, I would guess it will take you at least 1-2 days simply given the amount of data that needs to be copied during the process... so, talking about value of time - with approach I am suggesting, we will save couple of days of your time! While those extra seconds we are talking about probably won't have much impact at all considering 1-2 days timeframe to restore.

Alternatively, if you prefer doing backups for any reason, consider creating 2 PowerShell scripts which do automated restoration of all VMs from another site to current site. This would be even more robust solution, as all you would have to do to restore is run the script... no wizards at all. Even someone completely unaware about backups and restore will be able to launch this script, if you are away on vacation or something.

[tsightler]

Anton,

I'd like to interject into this thread that I'd really love to see a "multiple VM restore" option as well. As a person who recently used Veeam to recover from a disaster, I can say that walking through the wizard multiple times to restore 35 machines was one of the most annoying processes of the entire DR scenario. We have a strict priority of machines that should be restored and it would have been nice to have a GUI where I could simply pick the machine, their restore location, and hit "GO" and have it do it's work so that I could go about other pressing issues during a restore. Instead I'm sitting there, babysitting my backup product trying to keep 3-5 restores running in parallel, manually walking through a multi-step wizard. I cursed the Veeam GUI multiple times that night.

The basic problem is, Veeam as it's currently programmed feels like a product targeted at smallish datacenters, and perhaps that's true. The overall Veeam UI is quite nice if you have 10-20 VM's, but starts feeling cumbersome as the number of hosts and VM's grow. I understand that the use case of restoring 35 VM's (or 150 VM's like the OP) might be the exception rather than the rule during normal operations, however, it's during these disasters that a product really shines, or becomes an annoyance. In my case, the core Veeam product did it's job, we were able to restore our VM's, but instead of assisting me with the tasks, it was mostly annoying me with a silly wizard 35 times. It wasn't a showstopper for us, but if I had stepped through that wizard 150 times, I'd probably be looking for another product right now myself.

[Gostev]

Tom, I don't disagree this is nice feature to have. You are correct that situations like this are rather exception from rules, which is exactly why this feature did not have high enough priority before. There are workarounds for bulk restores (PowerShell), and talking about larger customers needs, they simply integrate bulk restores into their DR runbook automation scripts. Trust me, we have customers with as many as 1000 ESX sockets, and this feature is definitely not something they are begging us to deliver.

Anyhow, I am not trying to imply that this feature will not be implemented any time soon. In fact, it is planned for later this year as a part of major enhancements to the restore process (there is a number of common request we need to address). I am just trying to confirm my thinking that this is definitely not a showstopper (as you agreed). Besides, there are workarounds to provide even more robust solution for faster restores than competitive products provide today with better restore wizards.

I am sorry this is not in the product today - it all comes down to priorities and value. Just to illustrate how priorities work with example you will understand - I think you would agree that instant Linux file level restore feature our product provides is much more important than what we are discussing. Operational file level restores happen almost daily, unlike entire production restores like you have experienced (as you said before, this was first time in your life). Yet, Linux FLR is something competition does not provide.

Or, take my replication suggestion above... wouldn't you say that having built-in replication is tons more valuable in scenario Troy describes (when there are 2x spare capacity maintained in both sites). Again, our competition does not provide built-in replication.

Based on this, do you generally agree with how we've been prioritizing and selecting features to add to our product to this day?

[TroyB]

Anyhow, I am not trying to imply that this feature will not be implemented any time soon. In fact, it is planned to come later this year as a part of major enhancements to the restore process (there are a number of common request we need to address).

Wow. To think this was all I was looking for before all of this mess.

I would clarify why we can't just use replication, and also why we choose not to bother with Powershell at this time, but quite frankly, I've got more important things to do than continue to go back and forth with you when rather than just simply telling me "It's coming soon", you chose to pick apart my reasoning for wanting to do things a certain way when you know nothing about our environment at all.

I got the answer I was looking for. I might check back later when this feature is implimented.

[tsightler]

Well, I even thought about the suggestion of using replication for the OP's scenario, but then I realized that using Veeam replication wouldn't significantly change the behavior since you have to walk through the same wizard to promote a replica. Also, I'm not sure I would trust Veeam to replicate 150 VM's for a DR scenario as the chances of a disaster striking while a replication was in process would be too high and in this scenario you can't just "power on the VM's" you have to use the Veeam wizard to revert to the previous rollback.

The idea of scripting the restores is nice, but has it's own challenges. Scripted restores would require significant skill, testing, and maintenance for every change in your environment, including when Veeam is upgraded, when vCenter is upgraded, when storage is upgraded, perhaps even when hosts are added/removed. If your script only covers the most likely scenario then Murphy will make sure that the failure you experience is the one your script doesn't cover. I love scripting, and we use it heavily, but writing a script to cover every possible DR restore scenario seems like a nightmare for my environment.

I'm not really in a position to say whether Veeam selects it's priorities well as that's a matter of perspective. For me Linux FLR was important, for others, probably not so much. If Veeam didn't have Linux FLR I could still get the job done by restoring the VMDK and mounting them on another Linux VM, but it would be more annoying. Having recently experienced a DR restore my perspective is that Veeam could have made this task much easier, but instead it was annoying. If I find a product that has the features I need and includes more advanced restore functionality, it's going to get a look. Then I'll decide if Veeam is acceptable for my organization, but that's unlikely to be the same answer for another organization.

Overall, I would say Veeam has done a good job balancing features, but I do think Veeam has tons of room for improvement on the management of jobs, both backups and restores as those functions are generally inferior to other products I have used so these are simply areas where Veeam needs to show a willingness to improve. I didn't mean to say that Veeam wasn't doing that, only stating that I understand the OP's concern having just gone through a DR experience with Veeam myself and I can see how a DR restore wizard would be a nice added feature.

[Gostev]

I would clarify why we can't just use replication, and also why we choose not to bother with Powershell at this time, but quite frankly, I've got more important things to do than continue to go back and forth with you when rather than just simply telling me "It's coming soon", you chose to pick apart my reasoning for wanting to do things a certain way when you know nothing about our environment at all.

Well, this is exactly what this community forum is intended for - to go back and forth to ensure we have clear understanding of your environment, and why things we provide today (like replication, PowerShell) do not work for some customers. Without this information, we will not be able to improve our offering (e.g. make replication useful for you). Without your feedback, we will implement our own understanding based on what we think and what other customers say - but this won't work for you because of some specifics of your environment, because we don't know something about it. But I appreciate that you have more important things to do, and do not have time to participate in this discussion. There's nothing we can do about it...

Sorry if it sounded to you like I am picking apart you reasoning, do not value your time, not taking your requests in consideration etc. as it is definitely not the case. I was merely relaying our current point of view about this feature to fire up healthy discussion on why current approach and some workarounds do not really work. You don't have to be so defensive, we are here to listen to you, but it is fact that real truths only comes out in deep-dive discussions like this... this is how we have been doing this on these forums past years, and it's been working very well - because as they say, truth only comes out in argument. No need to go too far for example - if you look up our last year discussions with Tom, you will see that many features from there are either already implemented, or are coming in v5.

[Gostev]

Thanks Tom for your clarifications, just saw your post.

[donikatz]

Sorry for restarting an old thread, but just had to comment that I'm SOOO looking forward to this in 5.0 as I sit here all day restoring VMs to a new test environment...