Backup vs VM Copy / transactional integrity

[Frosty]

I'm very new to both VMware and Veeam (less than 24 hours experience of the latter!) so am looking for some more experienced heads to provide me with a little bit of guidance.

MY ENVIRONMENT
-- one (1) VMware server running vSphere 4.0 which has four (4) guest Windows Server O/S running on it
-- two (2) stand-alone Windows Servers
-- a small 4TB SAN attached to one of the Windows servers (but not to the VMware server)
-- I intend to use 1TB portable USB HDDs for backup/recovery storage
-- we have a DR site being built which will have two small VMware servers running ESXi Free

We are in the early stages of a gradual migration to VMware. By this time next year I expect to have a 2nd VMware vSphere server in place and we will then have virtualised our remaining stand-alone servers. Once we have done that, and have rebuilt the SAN to work with both VMware servers, things will be much better. But I need some backup/recovery solutions in place in the meantime as we transition.

Since I understand that Veeam Backup v4.1 will not work with ESXi Free edition, I have been looking instead at using the VM Copy function instead of the Backup functionality. This seems to provide me with what I need from a DR perspective: namely, the ability to restore a set of VMware virtual disk images and boot them.

I just have a few questions:
(1) what downside (if any) should I expect from relying on VM Copy instead of Backup?
(2) are the virtual disk copies made with VSS transactional integrity, just the same as with the Backup function?
(3) ran a test last night to VM Copy one running server (SQL06) and a directory "sql06.grv.local(112)" was created ... what is the significance of the (112) appelation? ... will this number change each time I make a copy?
(4) we have one legacy Windows 2000 Server machine ... VSS doesn't work with 2000, so I am just wondering what the implications of this might be ... do you think there is any risk that copies made via VM Copy would not boot in our DR scenario?

I am planning for now to run all my backups in two stages:
(1) backup everything to the SAN; then
(2) replicate all the backed up data to the 1TB portable USB drive for offsite storage

If I need to do any ad-hoc restoring of data, I will have a copy on hand on the SAN. Anything more serious (e.g. loss of premises due to extended power outages or fire or whatever) we will use the offsite USB drives to recover in our DR site.

That's about it. I would really welcome some feedback on this strategy.

[Gostev]

Hello, welcome to the world of VMware

1. Unfortunately, VM Copy is not supported on free ESXi either. Per product's System Requirements found in the Release Notes, free ESXi is completely unsupported by our product.
2. Yes, that is correct.
3. I am guessing that this is unique VM ID appended to prevent possible conflicts with similarly named VMs. I just need to double check with developers.
4. This depends on what applications are running there on Windows 2000.

You strategy sounds good, and it is quite typical!

[Frosty]

Thanks Gostev,

Re: (1) ... when you say that "VM Copy is not supported on free ESXi either" ... does that mean:

(A) that even if we copy the VM disk files created on our SAN by a scheduled VM Copy task and put them on a USB HDD and then take that to our DR site and copy those files to the free ESXi server ... that they won't boot?

or did you mean:

(B) that we just can't use Veeam Backup v4.1 to VM Copy those files to a free ESXi server, but if we use another tool to manually copy those files on the Free ESXi server they will be okay?

Reason I ask is that in our DR scenarios, we would be quite happy with scenario (B) as we hope that this process will never be required. If we need to load up VMs to recover individual files, or to fix a server crash, then we would normally be copying to a fully licensed vSphere v4.0 server, and in that scenario we know it will work okay.

[Gostev]

Stephen, I mean that the VM Copy job will fail out if you attempt to process VMs running on free ESXi server with the job. In other words, source ESXi server cannot be free.

[Frosty]

Cool ... thanks! ... I think we're okay with that then ... we will VM Copy the files off our licensed vSphere 4.0 server(s) and store to SAN and to USB HDD ... in a disaster scenario, we will manually copy these files from USB HDD to DR VMware free ESXi servers and boot them.

We'll run a test of that scenario in a few weeks, post-Christmas and New Year, just to be sure it is going to work okay.

Re: my other query, about the directory named "sql06.grv.local(112)" ... I have re-run that VM Copy job and it re-used that same directory name, it didn't create a new one with a different number suffix ... so it looks like your supposition about it being a VM ID is probably correct. As long as it doesn't change from backup to backup I'm okay (otherwise it would rapidly chew up my SAN storage).

Everything is looking pretty good here then! I just have to:

(1) time and sequence the VM Copy jobs and replication from SAN to USB HDD, so it all runs happily overnight; and
(2) run a test with our DR site so I can prove the ability to recover the data and boot the VMs there

Performance is not particularly great at the moment, as due to our current configuration I am basically having to copy all the VMs over the network without any benefit of the block tracking features. I will have to wait for additional hardware and an architecture upgrade here before all that will work. But at least the network is 1Gb/sec, not 100Mb/sec!

Thanks for your help!

[Gostev]

OK, now that I better understand what are you planning to do, I suggest that you use Backup jobs instead (for incremental backups, changed block tracking, compression/dedupe, rollbacks and all other cool stuff).

When the time comes to restore, instead of restoring directly to ESXi free (which is not supported because write operations via APIs are restricted for free ESXi), you could restore all VM files from backup (second option in Restore wizard); then upload those files to ESXi free by "some other means" and register VM manually.

[Frosty]

Because we don't have our SAN storage directly available to the vSphere server yet (all our SAN storage is formatted with NTFS and is connected to a stand-alone Windows server, not to a virtual server), I was under the impression that the vStorage API side of the Veeam software would not be fully functional.

I've run a couple of tests with the VM Copy function and no matter what options I select (vStorage vs Veeam Network) everything seems to run at pretty much the same speed. I got pretty much the same result yesterday with using backup. I assumed that its because the software is pulling the whole server image across the network to my stand-alone server and is then comparing it with the existing virtual disk or backup images stored on the SAN.

[Gostev]

You will be able to use vStorage API mode with "Network" option. This would work for all types of ESX storage, including local disks. So SAN is not required for you to be able to leverage the benefits of changed block tracking.

VM Copy job, on the other hand, never does incremental runs (it was just never designed with backup in mind - there are other use cases, so always does the full copy), thus changed block tracking is never leveraged for VM Copy jobs. Which was one of the reason why I suggested using Backup jobs instead.

[Frosty]

Fair enough, I will look into that. One of the other things on my "to do" list is to set up the virtual HDDs in the VMware server so that they are Thin Provisioned. That alone will probably cut hours off the backup jobs.

So much to do, so little time ... Christmas (and New Year and my annual leave!) is coming up quick!

Cheers, and thanks for the helping hand!

[Frosty]

Well Christmas, New Year and Annual Leave have all been successfully transacted. Back to the dull grind of work!

Just before I went on leave I completed the initial setup of our backup regime. It has run flawlessly whilst I was away, so that was a great outcome. Because we have a mix of virtual and non-virtual servers, our current backups comprise:

-- a DC running 'live' down in our DR site
-- a couple of physical servers covered by Windows backup to SAN
-- Veeam backup for the virtuals, configured to use VM Copy each night to SAN
-- batch file to replicate backup files from SAN to portable USB HDD for offsite storage

A bit clunky, and a bit inefficient ... but its working, so that's 90% of the battle. I just wanted to follow up on some of the earlier suggestions in this thread, and also to ask whether there are any ways to make a VM Copy job do its copying simultaneously to 2 locations ... e.g. to both the SAN and to the USB HDD ... this would probably reduce my backup elapsed time a fair bit.

Given that we want to backup to both SAN and USB HDD, if I wanted to switch from VM Copy to VM Backup, what would be the implications and what would be the 'correct' way to get the VM Backup files off the SAN and on to the USB HDD in a reliable way?

[Frosty]

I have been corresponding with people about VMware licensing. I've been informed 2nd-hand from contacts at VMware that as long as our DR site is not actively running any guest O/S but is only used as an emergency setup if our main VMware servers fail and are offline, then I am okay to re-use the "live" VMware licenses on the "DR" servers. Based on that advice, I will be able to restore direct from my USB drives to the licensed DR VMware server which will save a lot of time. I'm therefore in the process of switching from VM Copy to VM Backup (which I mentioned in another thread).

[Gostev]

Stephen, thanks for update - this is very useful information about VMware licensing indeed! I recall many people concerned about having to buy extra licenses for cold standby ESX hosts which will not be actually running any VMs unless disaster happens.

[Frosty]

It might be worthwhile Veeam making an enquiry with VMware along those lines, just to make sure that the information that I have been given is correct (it is, after all, somewhat 2nd hand, so it hasn't come to me directly from VMware). Here's a link to the discussion on the VMware boards, so you can read for yourself what I have been told:

http://communities.vmware.com/thread/257839?tstart=0

[Gostev]

I trust vExpert's word... but I will double-check with VMware directly if you wish.