Help about replication and SQL servers needed

[rfn]

Hi,

I'm hoping for some advice from all you experts out there. First a little story about our setup: We had used ESXi free edition on two hosts with local storage for about a year and a half when I wanted more security in our setup. The solution was to buy two EqualLogics, a SAS based PS4000XV and a SATA based PS4000E for replication in a secondary server room in another building.

I had hoped to use snapshotting as a means of backup and cancelled our remote backup solution, but even though I had explained this very clearly to Dell then they forgot to mention the pitfalls. The huge block size of EqualLogic storage (64 MB) made our snapshots take way more space than planned so this didn't work. Enter Veeam Backup and we recently got our license and have experimented with it.

Now, I can still use the replication feature in EqualLogic and even use the ASM/VE to create hypervisor aware snapshots, but they won't be transaction-consistent and we have a very important SQL server that I need to be sure will work in our replicated environment. I could opt to use Veeam for replication since their VSS integration would ensure that SQL server is replicated properly, but we only have a VMware Essentials license with 3 hosts and I need all three hosts in our production, and I'm unsure how much load it would take on a host to be the replication server.

The big question is if we really need replication with VBR v5? Now that it's possible to start a VM directly from backup then I'm wondering if we could just create backups hourly instead of daily and store the backup on the PS4000E in the secondary server room? In case of a major disaster in our primary server room or a failure of the PS4000VX then we could be up and running pretty quickly, but probably with low performance but that's probably not a big problem. If we had a fire in our office building and the primary server room is gone, then I'm sure that people are happy knowing that we're still able to work and haven't lost much data.

So... how would you do this? We have two EqualLogics, Veeam Backup and Replication, three ESX hosts and an extra unlicensed ESXi host in the secondary server room. I have plenty of space on a server in the secondary server room to store Veeam backups and the storage can be presented as SMB or NFS. Our goal is to be able to be back in business pretty quickly (a few hours) in case of a major failure or catastrophe in our primary server room, and be sure that data is replicated or backed in a way so that it isn't corrupted (transaction-consistent). We must not loose more than one our of work so replication or backup should happen at least once every hour.

We only have 8 VM's in production right now, so this isn't the biggest VMware environment on earth .

Regards,
René Frej Nielsen

[tsightler]

The disadvantage to backups over replication is basically that, when running from a Veeam backup, I/O performance is going to suffer a reasonable amount and you can't just leave your systems running here. As Anton would describe, running directly from your backup is more of a "spare tire" approach just like those little spares in your car, they'll keep you from being stuck on the side of the road, but your not going to be driving 75MPH for 1000 miles on that little spare.

Replication allows you to actually power on the VM, even assuming you Veeam server is destroyed as well. I'd pretty strongly suggest that replication is the right answer for your setup. One question would be, why are you concerned about to load replication would have when your not concerned about backups? The load is roughly the same and actually replication might very well take less load.

[rfn]

I see your point and it's a very valid one... We don't have Storage vMotion so would be able at all to move the VM away from the Veeam NFS? Is it available when powered off?

My point was that if we were in a situation where we had to resort to start up the replicated VM's, then we would really be in big trouble anyway. In case of a fire then we would be missing personal computers, papers and other stuff that people need to start working, so I would have time to start up our DR site and people wouldn't be concerned about lower speeds. But if we can make it so that we would have full speed, or close to, then that would be better of course.

My problem with performance is that my tests have shown that a VM with 4 vCPU on one of our ESX 4.1 boxes quite a lot faster that the older physical server that I had created for Veeam Backup. The physical server is connected to the SAN network so is able to backup using SAN mode, but it only has two older single core Xeon processors. So to get maximum performance I need to run Veeam in a VM but then it takes resources away from the production ESX hosts and we only have three in our license. I haven't tested what resources Veeam takes when replicating but if it's a lot then it's not so good for our production VM's. Can Veeam Backup run on a unlicensed ESXi host? I know it can't backup an unlicensed host, but can it run on one?

By using EqualLogic replication we don't put any load on our ESX hosts, but our SQL server isn't being replicated in a proper way. To do that I have to connect the database drives to the VM by using iSCSI from inside the VM, so that EqualLogics Auto Snapshot Manager can perform the needed VSS operations, but such drives can't be backed up by Veeam and then we need to implement another backup solution just for that. I don't want to do that, because that adds complexity and we're a small shop with only one IT guy (me) and the less systems I have to maintain the better. What I could do was to replicate the SQL server with Veeam and the others with EqualLogic, but I guess that I would also need to replicate our domain controller (which is unfortunately also our file and print server) with Veeam and then it suddenly be busy and take away resources from the ESX host it's running from.

If replication isn't a very taxing task then I could of course just use the physical server because we don't generate so much data that it can't finish the backup during the night, far from it. It's only the initial full backup which would have taken a long time on that server.

[Gostev]

I am not sure if hotadd will work with Veeam Backup running in a VM on free ESXi host. I think physical server may be a better option, especially given that you already have it built, and connected into SAN fabric (so able to use direct SAN access mode, which is the best way to do VMware backups).

[rfn]

OK... I guess you have a point about hotadd if it's not working in the free edition. I guess that I should test that... I will also need to test replication on our physical server and see if it can keep up with the amount of data that has to be handled.

Will it require more of the physical server if set it to replicate more often? The amount of data will then be less each time, but I don't know if it has to perform any other taxing jobs for each replication session.

[Gostev]

Almost no extra taxing on CPU... the load is mostly defined by the amount of data that needs to be processed during the corresponding run.

[rfn]

I can confirm that HotAdd doesn't seem to work on ESXi free edition. It simply fails... I could however use SAN mode, but I will first test how things performs on our older physical server with replication enabled.

I had one issue when I installed Veeam v5 and that was the path for VBRCatalog. I couldn't install it on a SMB share as I wanted because that would place the files on a server on our secondary server room and not locally on a server that might be lost in case of a disaster. What does the Catalog contain and what is it used for? Would it be a big problem to loose this Catalog?

[Vitaliy S.]

Rene,

VBR Catalog is used in search queries, so there is no big deal if you accidentally lose it due to a disaster.

When you create a backup job with file indexing option enabled, Veeam Backup server creates a catalog, or index, of the VM guest OS files and stores index files on the Veeam Backup server in the C:/VBR Catalog/Index/Machines/[vm_name] folder.

Once the index is created and stored on backup servers, the indexing service on Enterprise Manager performs index replication — it aggregates index data for all VM image backups from multiple backup servers. This consolidated index is stored on the Enterprise Manager server in the C:/VBR Catalog/Index/ global catalog and is used for search queries.

Hope it helps!

[rfn]

Nice explanation... thank you!

[Gostev]

Additionally, index is stored with the backup file itself as well, so there is no way to loose index data completely.
Please check out sticky v5 FAQ for more information about indexing.

[rfn]

I have now tested replication for our mailserver and the backup server seems just as busy as when doing backup, just as you wrote. I can imagine that it will be very busy through the whole day, so wouldn't it be smart to disable replication in the hours that backup is performed? I will also try to disable deduplication and compression to see if that helps.

Doesn't all these snapshot operations put a lot of load on our ESX hosts and SAN? If we replicate all our VM's this way then there will be snapshots on the datatores most of the time and with my limited knowledge of these things that seems like a really bad thing. Doesn't that create some issues with locking of the filesystem?

[rfn]

Just thinking here... if I choose to run Veeam Backup in a VM on one our hosts and use VA mode (which I have tested with great results) and enable replication, wouldn't I get great performance if I choose the same host as the replication target? Then the network traffic would only have to be transferred from the VM to the host itself and not through the LAN.

The replicated volumes would of course be stored on our secondary EqualLogic so data will be copied from one SAN to the other, but without entering the LAN.

[Gostev]

Yes, good thinking.

[rfn]

Well I have to be a little creative with our limited resources