Host-based backup of VMware vSphere VMs.
samuk
Expert
Posts: 127
Liked: never
Joined: Mar 18, 2009 2:15 pm
Full Name: Sam
Contact:

Testing Veeam Replication

Post by samuk »

Hi,

We are using V8 of veeam backup, and would like to test veeam replication.

Backups run at the moment every evening, and full backup on Friday evening.

Can the Veeam backups run along side the replication jobs? or will this cause issues?

as we are testing, we did not want to disable the backup jobs.

Any other pointers appreciated.

Have downloaded the user guide to read trough later today.

Thanks
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Testing Veeam Replication

Post by PTide »

Hi,

If you have two jobs trying to either backup or replicate the same VM at the same time, one of the jobs will wait until the first one ends. Please consider using post job script in order to avoid that problem.

Thanks.
samuk
Expert
Posts: 127
Liked: never
Joined: Mar 18, 2009 2:15 pm
Full Name: Sam
Contact:

Re: Testing Veeam Replication

Post by samuk »

Okay, thanks.

I have failed over, made a change and then failed back to production - onto existing VM.

We have 1 Veeam server (proxy) the DR ESX Host is in the same switch (on a different Vlan)

The fail back process is really slow - is this normal?

Code: Select all

28/09/2015 10:27:04          Failback started at 28/09/2015 10:27:04
28/09/2015 10:27:42          Queued for processing at 28/09/2015 10:27:42
28/09/2015 10:27:48          Preparing next VM for processing
28/09/2015 10:27:43          Required backup infrastructure resources have been assigned
28/09/2015 10:27:43          No available proxies are running on ESX(i) management interface subnet. Using proxies from a different subnet, performance may be impacted.
28/09/2015 10:27:48          Using source proxy VMware Backup Proxy [nbd]
28/09/2015 10:27:50          Using target proxy VMware Backup Proxy [nbd]
28/09/2015 10:27:52          Preparing original VM
28/09/2015 10:28:01          Creating working snapshot on original VM
28/09/2015 10:35:26          Calculating original signature Hard disk 1 (20.0 GB) 55% at 0 KB/s
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Testing Veeam Replication

Post by PTide »

How long does it take in total to failback? Please note, that during the failback to the original VM Veeam B&R needs to calculate and synchronize the differences between the original VM and the replica VM. The time required to scan the VM image depends on its size and is comparable to the full backup time of this VM. Also, in your case this
28/09/2015 10:27:43 No available proxies are running on ESX(i) management interface subnet. Using proxies from a different subnet, performance may be impacted.
means that source proxy pulls the data using network mode across WAN, instead of using hotadd mode and transferring data between proxies. Please check both target and source proxy availability.

Thank you.
samuk
Expert
Posts: 127
Liked: never
Joined: Mar 18, 2009 2:15 pm
Full Name: Sam
Contact:

Re: Testing Veeam Replication

Post by samuk »

Fallback took 15mins.

this was a small VM. Would veeam be okay with a 4TB VM?

Can i install a Veeam proxy on the DR VC? is there any licence implications? as the live environment is licensed.
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Testing Veeam Replication

Post by PTide »

Can i install a Veeam proxy on the DR VC? is there any licence implications? as the live environment is licensed.
Sure, that should speed up the process. The product components (proxies, backup repositories, backup servers) are not licensed separately. You can deploy as many of these components as you like while licensed to the correct number of sockets. So everything should go smooth.
this was a small VM.
How small was it?
]Would veeam be okay with a 4TB VM?
That depends on your connection and on how much data change will have happened to that VM. For example Exchange failback may take hours.
samuk
Expert
Posts: 127
Liked: never
Joined: Mar 18, 2009 2:15 pm
Full Name: Sam
Contact:

Re: Testing Veeam Replication

Post by samuk »

VM is 20GB.

I have now installed a backup proxy.

VM failed over, copied a small folder no more then 100mb, and failed back.

Process took 20 mins.

Code: Select all

28/09/2015 12:39:28          Failback started at 28/09/2015 12:39:28
28/09/2015 12:40:06          Queued for processing at 28/09/2015 12:40:06
28/09/2015 12:40:11          Preparing next VM for processing
28/09/2015 12:40:06          Required backup infrastructure resources have been assigned
28/09/2015 12:40:11          Using source proxy 10.216.4.51 [nbd]
28/09/2015 12:40:13          Using target proxy VMware Backup Proxy [nbd]
28/09/2015 12:40:16          Preparing original VM
28/09/2015 12:40:24          Creating working snapshot on original VM
28/09/2015 12:59:57          Calculating original signature Hard disk 1 (20.0 GB) 
28/09/2015 13:00:13          Replicating restore point for Hard disk 1 (20.0 GB) 767.5 MB processed
28/09/2015 13:00:16          Creating replica restore point
28/09/2015 13:00:56          Replicating changes Hard disk 1 (20.0 GB) 982.0 MB processed
28/09/2015 13:01:07          Removing working snapshot from original VM
28/09/2015 13:01:09          Powering on original VM
28/09/2015 13:01:26          Failback completed at 28/09/2015 13:01:26
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Testing Veeam Replication

Post by PTide »

28/09/2015 12:40:11 Using source proxy 10.216.4.51 [nbd]
28/09/2015 12:40:13 Using target proxy VMware Backup Proxy [nbd]
For some reason your proxies are still used in Network Block Device mode instead of hotadd. Please check with this article and the link provided in order to troubleshoot.

Thank you.
samuk
Expert
Posts: 127
Liked: never
Joined: Mar 18, 2009 2:15 pm
Full Name: Sam
Contact:

Re: Testing Veeam Replication

Post by samuk »

Okay looking into the link this morning.

I had just changed the proxy to use automatic mode rather then network.

the Failback is quicker then before...8 mins it took, last try was 15 - as 20 the time before that


Also @ the point of Calculating original signature Hard Disk 1 at 0 KB/S is this normal?

Code: Select all

29/09/2015 10:30:06          Failback started at 29/09/2015 10:30:06
29/09/2015 10:30:46          Queued for processing at 29/09/2015 10:30:46
29/09/2015 10:30:51          Preparing next VM for processing
29/09/2015 10:30:46          Required backup infrastructure resources have been assigned
29/09/2015 10:30:51          Using source proxy 10.216.4.51 [nbd]
29/09/2015 10:30:54          Using target proxy VMware Backup Proxy [nbd]
29/09/2015 10:30:56          Preparing original VM
29/09/2015 10:31:04          Creating working snapshot on original VM
29/09/2015 10:39:50          Calculating original signature Hard disk 1 (20.0 GB) 
29/09/2015 10:40:13          Replicating restore point for Hard disk 1 (20.0 GB) 899.0 MB processed
29/09/2015 10:40:16          Creating replica restore point
29/09/2015 10:40:50          Replicating changes Hard disk 1 (20.0 GB) 698.0 MB processed
29/09/2015 10:40:56          Removing working snapshot from original VM
29/09/2015 10:40:58          Powering on original VM
29/09/2015 10:41:16          Failback completed at 29/09/2015 10:41:16
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Testing Veeam Replication

Post by PTide »

I suppose that failback went faster this time due to a lesser amount of changes that had to be committed to an original VM. As you can see from this log
29/09/2015 10:30:51 Using source proxy 10.216.4.51 [nbd]
29/09/2015 10:30:54 Using target proxy VMware Backup Proxy [nbd]
your proxies are still using NBD mode. Please force both of your proxies to use Virtual Appliance mode so we can check if it's even available in your case. If it's not available then we will attempt to find the reason why.

Thank you.
samuk
Expert
Posts: 127
Liked: never
Joined: Mar 18, 2009 2:15 pm
Full Name: Sam
Contact:

Re: Testing Veeam Replication

Post by samuk »

Cool, just tried Virtual Appliance Mode - still says NBD - even thought network fail over is disabled.

Will carry on working my way through the link.
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Testing Veeam Replication

Post by PTide »

Cool, just tried Virtual Appliance Mode - still says NBD - even thought network fail over is disabled.
Looks like something went very wrong - please open a case with our support team so they can take a look at your logs and post your case ID here.

Thank you.
samuk
Expert
Posts: 127
Liked: never
Joined: Mar 18, 2009 2:15 pm
Full Name: Sam
Contact:

Re: Testing Veeam Replication

Post by samuk »

Done :-)

[ID# 01069765]
samuk
Expert
Posts: 127
Liked: never
Joined: Mar 18, 2009 2:15 pm
Full Name: Sam
Contact:

Re: Testing Veeam Replication

Post by samuk »

HI Just an update - think i need to explain the current environment:

2 environments SAN's are not linked.

LIVE Environment
3 ESX hosts (vsphere 5.5)
1 VC (VM)
1 Veeam Server & Proxy(Physical)
3 Par SAN

DR Environment
3 ESX Hosts
Veeam Proxy - Physical
EVA 3PAr SAN

Replication is occurring over the network - currently @ 5MB\s

Have exchanged afew emails with Vitaliy - and hot add is only supported if the Veeam proxy is installed on a VM on both locations.

Waiting for confirmation on this, and i will do this if everything else fits and will work.

have pointed him to this thread.
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Testing Veeam Replication

Post by PTide »

and hot add is only supported if the Veeam proxy is installed on a VM on both locations.
Correct! It's mentioned in this troubleshooting article:
The Proxy being used to back up the VM must be a Virtual Machine on a host that is able to access the datastore where the source VM’s disks are stored.
:)
samuk
Expert
Posts: 127
Liked: never
Joined: Mar 18, 2009 2:15 pm
Full Name: Sam
Contact:

Re: Testing Veeam Replication

Post by samuk »

Okay, so have installed proxy on a VM in the Live environment & in a VM on the DR environment.

Not sure what is meant by : The Proxy being used to back up the VM must be a Virtual Machine on a host that is able to access the datastore where the source VM’s disks are stored.

how do i give the proxy vm access to the datastores?


Also when i try to use the 2 new proxies to replicate i get this

01/10/2015 11:35:49 :: Task failed Error: No proxy available

So i had to disable the other 2 proxies, now is working

Got this even though proxy installed ok - is this to do with the live site or DR?

01/10/2015 11:44:00 :: No available proxies are running on ESX(i) management interface subnet. Using proxies from a different subnet, performance may be impacted.
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Testing Veeam Replication

Post by PTide »

Not sure what is meant by : The Proxy being used to back up the VM must be a Virtual Machine on a host that is able to access the datastore where the source VM’s disks are stored.
Your source proxy has to reside on the same ESXi host where replicated VM os located or, in case of shared storage, source proxy can reside on some other host that has an access to the storage. you can follow this guide to check if your infrastructure ready.

Both source and target proxy have to see each other in order to perform data transfer. Also each proxy has to be in the same subnet with ESXi host that has an access to replicated VM's vmdk.
samuk
Expert
Posts: 127
Liked: never
Joined: Mar 18, 2009 2:15 pm
Full Name: Sam
Contact:

Re: Testing Veeam Replication

Post by samuk »

HI,

just a quick update and afew questions.

Hot HDD looks like its working on only 1 esx host - confused as the hosts are identical and so are the datastores presented.



I have setup a replication job to happen every 30mins - this contains 6 VM's

Job log says 35 mins for the job to complete for the incremental jobs (Data transferred 1.2GB)

Little confused to if i can set the job to run every 15mins? or will this not work?

Also seeing source is busy 99% for all vm's being replicated..

Speeds are still poor? most of the time is used up processing?

Code: Select all

08/10/2015 10:00:18 :: Job started at 08/10/2015 10:00:17 
08/10/2015 10:00:19 :: Building VM list 
08/10/2015 10:01:23 :: VM size: 866.0 GB 
08/10/2015 10:01:23 :: Changed block tracking is enabled 
08/10/2015 10:01:25 :: Preparing next VM for processing 
08/10/2015 10:01:30 :: Preparing next VM for processing 
08/10/2015 10:01:30 :: Processing Web2 
08/10/2015 10:06:40 :: Processing Srv01 
08/10/2015 10:06:40 :: Preparing next VM for processing 
08/10/2015 10:12:50 :: Processing SRV02 
08/10/2015 10:12:50 :: Preparing next VM for processing 
08/10/2015 10:21:01 :: Preparing next VM for processing 
08/10/2015 10:21:02 :: Processing Srv03 
08/10/2015 10:25:16 :: Processing Srv01 
08/10/2015 10:25:16 :: Preparing next VM for processing 
08/10/2015 10:29:01 :: Processing king8 
08/10/2015 10:29:01 :: All VMs have been queued for processing 
08/10/2015 10:35:10 :: 1 restore point removed by retention policy from VM Web2_replica 
08/10/2015 10:35:18 :: 1 restore point removed by retention policy from VM Srv01_replica 
08/10/2015 10:35:26 :: 1 restore point removed by retention policy from VM SRV02_replica 
08/10/2015 10:35:38 :: 1 restore point removed by retention policy from VM Srv03_replica 
08/10/2015 10:35:44 :: 1 restore point removed by retention policy from VM Srv01_replica 
08/10/2015 10:35:50 :: 1 restore point removed by retention policy from VM king8_replica 
08/10/2015 10:35:59 :: Load: Source 97% > Proxy 30% > Network 0% > Target 23% 
08/10/2015 10:35:59 :: Primary bottleneck: Source 
08/10/2015 10:35:59 :: Job finished at 08/10/2015 10:35:59
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Testing Veeam Replication

Post by PTide »

Hot HDD looks like its working on only 1 esx host - confused as the hosts are identical and so are the datastores presented.
That's strange. Please keep working with support on that.
Data transferred 1.2GB<...>Little confused to if i can set the job to run every 15mins? or will this not work?
What's he total size of those 6 VMs?
Also seeing source is busy 99% for all vm's being replicated..
What kind of production datastore do you have and how is it connected to ESXi?
samuk
Expert
Posts: 127
Liked: never
Joined: Mar 18, 2009 2:15 pm
Full Name: Sam
Contact:

Re: Testing Veeam Replication

Post by samuk »

Total size of the VM's is 866GB

Have several VMFS5 datastores, connected to the ESX hosts over Fibre (Round Robin configuration within Vsphere 5.5


Is the source the ESX host or the Veeam backup server?
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Testing Veeam Replication

Post by PTide »

Is the source the ESX host
This. More specifically - the datastore (btw, what's the vendor, model?). Please check your datastore performance.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Testing Veeam Replication

Post by foggy »

Sam, what is the processing rate during the initial (full) job run?
samuk
Expert
Posts: 127
Liked: never
Joined: Mar 18, 2009 2:15 pm
Full Name: Sam
Contact:

Re: Testing Veeam Replication

Post by samuk »

HP 3Par SAN -

processing time for the original job - how can i get this out, as the job report does not tell me.



Veeam support advised the hotadd is not in the scope of veeam support - as its working on one host and not the others. they have still offered to help, however our go live date is today - this is now getting urgent.

2 esx hosts not able to replicate.

I have also logged a call with VMware - who said to review this article: http://kb.vmware.com/kb/1008524

So little confused to what and how to get this resolved...


Any thoughts?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Testing Veeam Replication

Post by foggy »

You can use arrow keys in the job stats window to switch between job runs.

Why not to try direct SAN access, since you seem to have physical Veeam B&R server?
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Testing Veeam Replication

Post by PTide »

Hot HDD looks like its working on only 1 esx host - confused as the hosts are identical and so are the datastores presented.
If two identical things work differentIy then I believe that you should to either double check everything, or escalate the case to get it passed to a higher tier. Do to the latter please click "Talk to a Manager" button on the customer support portal. Sorry, but not much we can do on the forum in your case
samuk
Expert
Posts: 127
Liked: never
Joined: Mar 18, 2009 2:15 pm
Full Name: Sam
Contact:

Re: Testing Veeam Replication

Post by samuk »

Indeed you can move through using the arrow keys.. :-)

I will report back Monday. I have requested this case be escalated

I am able to download the VMX if i connect to the host directly using the VI client, but not if i connect to the Vcenter server - vmware say this is by design?

Also noted, replication is not working on the other hosts over the network.

I ran the same incremental job over the network, and it seemed to have worked quicker then hotadd?

09/10/2015 16:19:10 :: Load: Source 96% > Proxy 37% > Network 4% > Target 17%
09/10/2015 16:19:10 :: Primary bottleneck: Source

Took 20mins, rather then 35 mins using hottadd..

Will do more testing over the weekend
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Testing Veeam Replication

Post by foggy »

Sometimes hotadd can take longer due to overhead required to actually hotadd disks.
samuk
Expert
Posts: 127
Liked: never
Joined: Mar 18, 2009 2:15 pm
Full Name: Sam
Contact:

Re: Testing Veeam Replication

Post by samuk »

Still finding Network mode is quicker then Hotadd... Any thoughts?

How many VM's is recommended but replication job? have 25VM's in total, hoping to run replication every 15mins

Also what do you make of this? Says no proxy on the Managment but has it listed as in use

Code: Select all

12/10/2015 15:19:32 :: Queued for processing at 12/10/2015 15:19:32 
12/10/2015 15:19:32 :: Required backup infrastructure resources have been assigned 
12/10/2015 15:19:32 :: [b]No available proxies are running on ESX(i) management interface subnet. Using proxies from a different subnet, performance may be impacted. [/b]
12/10/2015 15:19:37 :: VM processing started at 12/10/2015 15:19:37 
12/10/2015 15:19:37 :: VM size: 20.0 GB 
12/10/2015 15:19:42 :: Discovering replica VM 
12/10/2015 15:19:43 :: Getting VM info from vSphere 
12/10/2015 15:19:52 :: Creating VM snapshot 
12/10/2015 15:19:58 :: Preparing replica VM 
12/10/2015 15:20:15 :: Processing configuration 
12/10/2015 15:20:22 :: Creating helper snapshot 
12/10/2015 15:20:26 :: [b]Using source proxy MS01 for disk Hard disk 1 [nbd] [/b]    (Source Proxy)
12/10/2015 15:20:26 :: Using target proxy VC01 for disk Hard disk 1 [nbd] 
12/10/2015 15:20:31 :: Hard disk 1 (20.0 GB) 24.0 MB read at 16 MB/s [CBT]
12/10/2015 15:21:01 :: Removing VM snapshot... (100% done)
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Testing Veeam Replication

Post by PTide »

Have you tried to login into proxy and ping your ESXi's management interface from there?
samuk
Expert
Posts: 127
Liked: never
Joined: Mar 18, 2009 2:15 pm
Full Name: Sam
Contact:

Re: Testing Veeam Replication

Post by samuk »

Yes, and it could not ping one of the hosts - this is now resolved and i am no longer getting the NFC error when replicating.

Have afew more questions:

I had setup afew replication jobs to use the Network mode (as hot add is still not working, need to get more logs to support)

I had an issue with replicating a 1.5TB VM:

14/10/2015 16:57:21 :: Processing Parking5 Error: Transmission pipeline hanged, aborting process.

Guessing this is a network issue?

Also on the incremental relication of 2 terminal servers, and users were not able to login for the duration of the replication.

How well does veeam cope with busy servers & network replication?
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 70 guests