Replication and backup of VM at the same time

[mephisto]

Hi there,

It possible to run a backup and a replication job of the same VM at the same time on ESXi 5.1 and Veeam 6.5?

I've got some big VMs I'm replicating for the first time over the WAN (2TB) and I don't want to disable the backup jobs because of the replication. Will I have problems running a backup job meanwhile the replication is working?

Regards,

Alex

[dellock6]

A VM can be processed by a single job at the same time, if another job wants to process the same VM it will be placed in the queue waiting for the first job to complete.
If you already have backups of that VM, you can more simply move that backup out-of-band to the other site with a simple external disk, import it in the replica site and use it as a seed for the replica job. In this way even the first replica job will be incremental instead of full.

Luca.

[mephisto]

They are 2 different datacentres far from each other, no way to do a seed backup.

I'll need to rely on the line for this.

Thanks for the tip anyway

[dellock6]

ok, so you can ship via the line a backup file, so the VM will not be locked for the time of the copy, and also you can use some filecopy software able to do resume, so in case of transmission interruptions you would have the ability to resume it, while the first full replica needs to go from start to end to have a good replicated VM.

Luca.

[mephisto]

ok, so you can ship via the line a backup file, so the VM will not be locked for the time of the copy, and also you can use some filecopy software able to do resume, so in case of transmission interruptions you would have the ability to resume it, while the first full replica needs to go from start to end to have a good replicated VM.

Luca.

Can I transform this backup job transmitted into a replicated VM?

The destination needs to be in a position of being ready to accept replication copies from the original VM

[veremin]

Step-by-step process regarding Replica seeding:

1. Create a backup of a VM that you're willing to replicate. (You've done it already)
2. Copy the backup-file from repository in the production site to the backup repository in the DR site.
3. While creating a replication job, you should point it to the backup repository in the DR site. During the first run of a replication job, Veeam Backup & Replication accesses the repository where the seed (the backup of a VM to replicate) is located, and restores the VM from the backup.
4. After that, VB&R synchronizes the state of restored VM with the latest state of VM residing in production site. After that you will have two restore points, one is the state of VM from the backup file, the other is latest state of VM in Production site.

Otherwise, you can transfer backup file, perform the restoration process, and later during creation of the job map Production VM to the already restored DR one.

More information can be found in the corresponding User Guide.

Hope this helps.
Thanks.

[mephisto]

I'll do this tomorrow with the other VMs as I'm half way through the replication of one already. Thanks for the step by step, much appreciated!

Quick question about the "free space" on the VMDK disks:

I've got a VM that has 20GB of data on a system disk (30GB disk), plus other partition that has 5GB of data (100GB disk) and during the replication so far I've noticed 24GB send over the line. I was expecting the free space of the disk being read right quick as there is no data on it and accelerate the speed rate as no real data is being transmitted, but still around 3MB/sec (20mbit link using all sort of maximum compression options on veeam). What do you guys think that is happening on my case? Seems like it is still sending data over the line from these disks even though the data set itself not compressed is around 25GB.

Any ideas?

Thanks!

[mephisto]

Just another quick on this, so I'll transfer to destination repository (Windows VM) and then from there the replication job will use that data as base to build the proper replicated VM + the synced data on the VMware datastore, right? Just clarifying this as I need to plan on storage as I'll sort of have 2 copies of the VM so it can use quite a lot of space. Thanks!

[ZachW]

The VM will be restored from the backup file to a datastore specified in at the DR site. Once this has been restored a task called "calculating disk digests" will be performed between the source VM and the restored VM. Once the disk digests have been calculated the changed data will then be replicated over. This will show as two different tasks in the first run of the replication after pointing to the seed. One task will show ( in the real-time statistics ) as VMname (seed) and the next will just show as VMname.

[foggy]

I've got a VM that has 20GB of data on a system disk (30GB disk), plus other partition that has 5GB of data (100GB disk) and during the replication so far I've noticed 24GB send over the line.

Is it a full or incremental pass?

but still around 3MB/sec

Are you talking about the data transfer speed or speed of data retrieval from source (reported in the job session log)?

[mephisto]

I think CBT is kicking in, the VM has actually 120GB of data and the backup turned out to be 220GB, I want to clone it as then the thindisks on the cloned machine will get way smaller. This machine had lots of data deleted today.

Would I be able to clone the VM, upload to the DR site and then use the cloned one as a base for replication of the original VM? Would I encounter problems with the replication because it was a cloned copy that was uploaded? Thanks!

[veremin]

Would I be able to clone the VM, upload to the DR site and then use the cloned one as a base for replication of the original VM? Would I encounter problems with the replication because it was a cloned copy that was uploaded?

In the way I see it, this scenario should work fine without any issues. However, be aware that it isn’t the only way to preserve disk-provisioned type. Veeam Backup & Replication saves the format of replicated VM disks by default. If need be, the disk format can be configured within the job. For instance, if production VM is using thick disks, you can change the format of replica disks to thin provisioned, it, in its case, will result in saving on disk space necessary for storing replica data.

As to the CBT, may I ask you to confirm that aforesaid VM doesn’t have snapshots at all, since CBT doesn’t work while backing up VM with at least one snapshot. Additionally, have ever seen this article describing VMware CBT limitations?

Furthermore, are the figures you’ve mentioned from incremental or full backup run? What backup mode is used? What are the storage optimization settings? Is this a particular VM with high change rate (SQL, Exchange, etc)? Are there any antivirus and/or defrag activities being performed at this server?

Moreover, it might worth checking this topic for potential answer.

Hope this helps.
Thanks.

[foggy]

I think CBT is kicking in, the VM has actually 120GB of data and the backup turned out to be 220GB, I want to clone it as then the thindisks on the cloned machine will get way smaller. This machine had lots of data deleted today.

Any block modification inside the guest OS including deleting files results in changing virtual disk blocks, so obviously CBT marks these blocks as new ones.

[mephisto]

Well I deleted lots of data from the VM, cloned is as thinprovisioning and is still using 400GB+ :/

I really don't get this, normally I would delete data from a VM that is using thindisk, and after cloning of vmotion it to different storage I would claim my space back. Now the VMDKs are still quite big, not really sure why as the VM now has 130GB of actual data inside it, and the Veeam backup comes out compressed as 240GB, the VM is using 400GB+ for some reason and I can't really troubleshoot what is going on :/

[mephisto]

Is it a full or incremental pass?
Are you talking about the data transfer speed or speed of data retrieval from source (reported in the job session log)?

Full backup (replication)

I was taking the numbers from the statistics of the running job, processing rate.

[mephisto]

In the way I see it, this scenario should work fine without any issues. However, be aware that it isn’t the only way to preserve disk-provisioned type. Veeam Backup & Replication saves the format of replicated VM disks by default. If need be, the disk format can be configured within the job. For instance, if production VM is using thick disks, you can change the format of replica disks to thin provisioned, it, in its case, will result in saving on disk space necessary for storing replica data.

As to the CBT, may I ask you to confirm that aforesaid VM doesn’t have snapshots at all, since CBT doesn’t work while backing up VM with at least one snapshot. Additionally, have ever seen this article describing VMware CBT limitations?

Furthermore, are the figures you’ve mentioned from incremental or full backup run? What backup mode is used? What are the storage optimization settings? Is this a particular VM with high change rate (SQL, Exchange, etc)? Are there any antivirus and/or defrag activities being performed at this server?

Moreover, it might worth checking this topic for potential answer.

Hope this helps.
Thanks.

All VMs are in thinprovisioning at the moment for this client, strangely I'm having problems claiming the free space back after deleting data inside the VMs. Would be because the Vms where converted from physical and there were a few partitions inside each physical disk? Could this be causing the problem about not being able to claim the free space back?

[foggy]

Well I deleted lots of data from the VM, cloned is as thinprovisioning and is still using 400GB+ :/

Have you run sdelete on the VM volumes prior migrating it?

I was taking the numbers from the statistics of the running job, processing rate.

What transport mode is being effectively used during processing (this can be found in the job stats window right after the name of the proxy server)?

[mephisto]

Any block modification inside the guest OS including deleting files results in changing virtual disk blocks, so obviously CBT marks these blocks as new ones.

What about when I create a full backup of this, like with replication.

I assumed it would be a rather small backup as lots of data have been scraped and a full backup job was started.

[veremin]

As Alexander has already mentioned, in the case where you previously had data on the drive and deleted it and wish to reclaim the space, it’s necessary to first zero out the blocks which had been previously written to the drive and only then start the migration process.

In Windows OS: the easiest way to do that in is to use a tool like SDelete. SDelete is capable of scanning a drive and zeroing out blocks from deleted files so that they can be reclaimed as free space. Use the following command to zero out the free space:

Code: Select all

Sdelete.exe -"parameter" c:\ <-- Replace the c:\ with the appropriate drive letter

Note: The "parameter" which is used for zeroing free space in SDelete utility depends on its version. In Secure Delete v1.51 "-c" is responsible for such a task. In Secure Delete v1.6 (and newer) this role is played by "-z".

In Linux OS: for zero out free space you can use this:

Code: Select all

dd if=/dev/zero bs=64k of=zero.temp ; rm zero.temp

Note: don’t forget to implement the command to each partition you have.

Hope this helps.
Thanks.

[mephisto]

Have you run sdelete on the VM volumes prior migrating it?
What transport mode is being effectively used during processing (this can be found in the job stats window right after the name of the proxy server)?

I didn't use sdelete, I just realised that when you vmotion the vm to a storage that has the same blocksize, it won't shrink the disks. So yeah, I'll need to use sdelete, never used it before, thanks for the tip

transport mode? Not really sure where I can check this, but I'm using Extreme compression and WAN target if that helps?

[mephisto]

As Alexander has already mentioned, in the case where you previously had data on the drive and deleted it and wish to reclaim the space, it’s necessary to first zero out the blocks which had been previously written to the drive and only then start the migration process.

In Windows OS: the easiest way to do that in is to use a tool like SDelete. SDelete is capable of scanning a drive and zeroing out blocks from deleted files so that they can be reclaimed as free space. Use the following command to zero out the free space:

Code: Select all

Sdelete.exe -c c:\ <-- Replace the c:\ with the appropriate drive letter

Note: The "parameter" which is used for zeroing free space in SDelete utility depends on its version. In Secure Delete v1.51 "-c" is responsible for such a task. In Secure Delete v1.6 (and newer) this role is played by "-z".

In Linux OS: for zero out free space you can use this:

Code: Select all

dd if=/dev/zero bs=64k of=zero.temp ; rm zero.temp

Note: don’t forget to implement the command to each partition you have.

Hope this helps.
Thanks.

Thanks for that, much appreciated.

I've also found that I need to run this after Sdelete to shrink the disks, could you please confirm?

Code: Select all

vmkfstools -K /path/to/disk-name.vmdk

[foggy]

transport mode? Not really sure where I can check this, but I'm using Extreme compression and WAN target if that helps?

This information is available in the job statistics. Just select the particular VM to the left and you will see the transport mode tag right after the name of the proxy server effectively being selected for processing of that VM. It is one of the following: [san], [hotadd], or [nbd].

[mephisto]

This information is available in the job statistics. Just select the particular VM to the left and you will see the transport mode tag right after the name of the proxy server effectively being selected for processing of that VM. It is one of the following: [san], [hotadd], or [nbd].

Is it on hotadd

[veremin]

I've also found that I need to run this after Sdelete to shrink the disks, could you please confirm?

Code: Select all

vmkfstools -K /path/to/disk-name.vmdk

As far as I’m concerned, there is no need in running this command, If after having run sdelelete tool you’ll perform migration process.

Hope this helps.
Thanks.

[mephisto]

As far as I’m concerned, there is no need in running this command, If after having run sdelelete tool you’ll perform migration process.

Hope this helps.
Thanks.

But if I run the sdelete tool I will inflate those disks to full size, I need the original running VM to also shrink down in size. I would need to run this command to shrink the original VM down, right?

[mephisto]

Have you guys ever used this tool?

http://labs.vmware.com/flings/guest-reclaim

[veremin]

According to VMware, the command vmkfstools followed by -K parameter allows you to convert a preallocated virtual disk to eager zeroed thick and maintains any existing data. This process isn’t the same as shrinking the size of thin disk.

Thanks.

[foggy]

I was taking the numbers from the statistics of the running job, processing rate.

Overall job processing rate is the average speed of processing of all VMs in the job (total processed data divided by the time the job took to finish). 20Mbit link allows for 2.5MB/s transfer at max so the number you see on your job (~3MB/s) is pretty much fair.

[mephisto]

According to VMware, the command vmkfstools followed by -K parameter allows you to convert a preallocated virtual disk to eager zeroed thick and maintains any existing data. This process isn’t the same as shrinking the size of thin disk.

Thanks.

Do you know how I would shrink the disks on the original VM then as the sdelete will inflate the disk to full size? Thanks!

[mephisto]

Overall job processing rate is the average speed of processing of all VMs in the job (total processed data divided by the time the job took to finish). 20Mbit link allows for 2.5MB/s transfer at max so the number you see on your job (~3MB/s) is pretty much fair.

Yeah, I understand this, but I was expecting it to process the "free space" way quicker but now I know whay it is not, it is still considering all of that deleted data as "actual data" and compressing/transferring it. So hopefuly after sdelete I should be able to shrink that down and Veeam will transfer only the actual data of the VM, not the deleted data all together.

Thanks!