-
- Influencer
- Posts: 10
- Liked: never
- Joined: May 26, 2014 4:26 pm
- Full Name: Michel Deschenes
- Contact:
How to confirm VEEAM v7 is using DirectSan?
Hi,
I just installed VEEAM v7.0.3 for the first time on a physical WIndows 2012R2 and connected on my vCenter infrastructure.
I have attached my LUNs as read-only to my server and followed the online guides to setup Direct San backups. I even force the repository to use only Direct San. Though I am unsure if it really is using it. I am getting 40mbps throughput and there is a lot of network activity on my Physical server and my ESX server. There is also very little disk IO on my SAN from the VEEAM server. How would I go to confirm that VEEAM is indeed using Direct San access?
I wondered if the SAN access was simply just slow but testing with IOMeter gives me 9000+ IOPS and 120mbps/worker thread.
VEEAM server: IBM HS21, 8 cores, 32gb ram, 4gbps qlogic hba
SAN : DELL Compellent FC
Thanks for any insight!
I just installed VEEAM v7.0.3 for the first time on a physical WIndows 2012R2 and connected on my vCenter infrastructure.
I have attached my LUNs as read-only to my server and followed the online guides to setup Direct San backups. I even force the repository to use only Direct San. Though I am unsure if it really is using it. I am getting 40mbps throughput and there is a lot of network activity on my Physical server and my ESX server. There is also very little disk IO on my SAN from the VEEAM server. How would I go to confirm that VEEAM is indeed using Direct San access?
I wondered if the SAN access was simply just slow but testing with IOMeter gives me 9000+ IOPS and 120mbps/worker thread.
VEEAM server: IBM HS21, 8 cores, 32gb ram, 4gbps qlogic hba
SAN : DELL Compellent FC
Thanks for any insight!
-
- VP, Product Management
- Posts: 27377
- Liked: 2802 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
Hello Michel,
You can confirm that by looking at your job session (pick any VM) and search for [san] tag in the backup job log. Can you please tell me what is your bottleneck stats for the backup job?
Also can you please clarify how you forced your repository to use direct SAN access?
Thank you!
You can confirm that by looking at your job session (pick any VM) and search for [san] tag in the backup job log. Can you please tell me what is your bottleneck stats for the backup job?
Also can you please clarify how you forced your repository to use direct SAN access?
Thank you!
-
- Influencer
- Posts: 10
- Liked: never
- Joined: May 26, 2014 4:26 pm
- Full Name: Michel Deschenes
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
Hi Vitaliy,
Ok, my backups is indeed using the SAN then, on to trying to figure out why it's slow..
Bottleneck is Source. Which would lead to SAN access being slow but IOMeter gives me an average of 120mbps/worker thread using 64kb random reads. Also, the read IOPS from my Veeam server is really low.
I forced Direct SAN access by going into the backup proxy setting and setting transport mode to "Direct SAN access" and unchecking the "failover to network mode" checkbox.
Ok, my backups is indeed using the SAN then, on to trying to figure out why it's slow..
Bottleneck is Source. Which would lead to SAN access being slow but IOMeter gives me an average of 120mbps/worker thread using 64kb random reads. Also, the read IOPS from my Veeam server is really low.
I forced Direct SAN access by going into the backup proxy setting and setting transport mode to "Direct SAN access" and unchecking the "failover to network mode" checkbox.
-
- VP, Product Management
- Posts: 27377
- Liked: 2802 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
Michel, do you have the MPIO software installed on the backup proxy? Are you running the latest version of the FC driver?
-
- Influencer
- Posts: 10
- Liked: never
- Joined: May 26, 2014 4:26 pm
- Full Name: Michel Deschenes
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
I have the Windows 2012 MPIO feature installed. I have installed the latest HBA driver from windows update. I could try updating to the latest version.
Thing to note is that my OS and Repository are both running on the SAN also.
Thing to note is that my OS and Repository are both running on the SAN also.
-
- VP, Product Management
- Posts: 27377
- Liked: 2802 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
Try temporarily disabling MPIO, as I've seen a couple of posts on these forums where our clients reported decreased direct SAN performance with MPIO enabled.
As regards having repository on the same SAN, then it is not the best practice to do so, as you definitely don't want to lose production VMs and your backups if your SAN goes down. I would recommend using backup copy jobs to offload your backups to some other storage device, just to be on the safe side.
As regards having repository on the same SAN, then it is not the best practice to do so, as you definitely don't want to lose production VMs and your backups if your SAN goes down. I would recommend using backup copy jobs to offload your backups to some other storage device, just to be on the safe side.
-
- Influencer
- Posts: 10
- Liked: never
- Joined: May 26, 2014 4:26 pm
- Full Name: Michel Deschenes
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
I will try disabling MPIO and see how that goes and will update.
I do plan to use a backup copy on LTO4 tapes when I move into production (I'm currently just trying to set it up to replace our existing backup solution).
I also am setting up replication jobs to our second datacenter.
I do plan to use a backup copy on LTO4 tapes when I move into production (I'm currently just trying to set it up to replace our existing backup solution).
I also am setting up replication jobs to our second datacenter.
-
- VP, Product Management
- Posts: 27377
- Liked: 2802 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
In this case your backup strategy looks good to me Please keep us informed on your findings after playing with MPIO parameters.
-
- Influencer
- Posts: 10
- Liked: never
- Joined: May 26, 2014 4:26 pm
- Full Name: Michel Deschenes
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
Just disabled MPIO from within Windows, no real performance change.
One thing I noticed is that if I run a job with 1 VM in it, I get ~150mbps on my storage. If there are 4 VMs, I get ~350 mbps. It's as if each thread ran by VEEAM was slow even though it says it's bottleneck is source :S
Veeam reported an average of 86mbps when running a job with only 1 VM, 200mbps when running a job with 4 VMs (dropping by about 50 mbps everytime a VM was fully backed up).
1 VM = 5/27/2014 10:55:42 AM :: Load: Source 99% > Proxy 28% > Network 4% > Target 0%
4 VMs = 5/27/2014 11:14:13 AM :: Load: Source 99% > Proxy 55% > Network 6% > Target 0%
One thing I noticed is that if I run a job with 1 VM in it, I get ~150mbps on my storage. If there are 4 VMs, I get ~350 mbps. It's as if each thread ran by VEEAM was slow even though it says it's bottleneck is source :S
Veeam reported an average of 86mbps when running a job with only 1 VM, 200mbps when running a job with 4 VMs (dropping by about 50 mbps everytime a VM was fully backed up).
1 VM = 5/27/2014 10:55:42 AM :: Load: Source 99% > Proxy 28% > Network 4% > Target 0%
4 VMs = 5/27/2014 11:14:13 AM :: Load: Source 99% > Proxy 55% > Network 6% > Target 0%
-
- Influencer
- Posts: 10
- Liked: never
- Joined: May 26, 2014 4:26 pm
- Full Name: Michel Deschenes
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
I just got a weird thing now.. I started backing up my file server which weighs about 6TB. My new bottleneck is network? How would that be when it's using Direct SAN Access?
-
- VP, Product Management
- Posts: 27377
- Liked: 2802 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
Michel, can you please post full bottleneck stats for this very job?
-
- Influencer
- Posts: 10
- Liked: never
- Joined: May 26, 2014 4:26 pm
- Full Name: Michel Deschenes
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
Took a while to complete.. 10TB had to be backed up at an average of 120mbps
5/28/2014 11:49:47 AM :: Busy: Source 86% > Proxy 54% > Network 34% > Target 0%
Bottleneck changed to Source again during the evening.. Weird that Network was busy though as it did say it was using SAN..
5/28/2014 11:49:47 AM :: Busy: Source 86% > Proxy 54% > Network 34% > Target 0%
Bottleneck changed to Source again during the evening.. Weird that Network was busy though as it did say it was using SAN..
-
- VP, Product Management
- Posts: 27377
- Liked: 2802 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
Just to make sure we are on the same page, are you referring to Megabits per second (Mbps) or Megabytes per second (MBps) for this job run? If it is megabytes, then you have a pretty decent performance for the full job run.
How did you configure your repository server? Do you have both roles of repository and proxy servers deployed on the same machine?michel_deschenes wrote:I just got a weird thing now.. I started backing up my file server which weighs about 6TB. My new bottleneck is network? How would that be when it's using Direct SAN Access?
-
- Influencer
- Posts: 10
- Liked: never
- Joined: May 26, 2014 4:26 pm
- Full Name: Michel Deschenes
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
Yes, my veeam server is also proxy and repository. I have the ESX LUNs mounted on it (read-only).
-
- VP, Product Management
- Posts: 27377
- Liked: 2802 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
If you have both proxy and repository server (pointed to the local hard disk) on the same backup server, then data transfer should be LAN-free. And what about the first question?
-
- Product Manager
- Posts: 20415
- Liked: 2303 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
Even when executed on the same machine, VB&R source and target components communicate with each other via TCP. The traffic stays local, though. This communication has been identified as the bottleneck in your case. Thanks.My new bottleneck is network?
-
- Influencer
- Posts: 10
- Liked: never
- Joined: May 26, 2014 4:26 pm
- Full Name: Michel Deschenes
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
Oh sorry I didn't see the first question.
I am talking about MBps, so 200MBps would be 1.6gbps.
My problem is that every single thread are slow. I get a max of 50MBps (400mbps) per thread (which drops to 30MBps (240mbps) when processing a very large VM). Why can I higher speed when running multiple threads but 1 thread is stuck at a max of 400mbps. Storage is barely using and IOMeter gives me much better performance using 100% random read (125MBps (1gbps)) on a single thread.
I am talking about MBps, so 200MBps would be 1.6gbps.
My problem is that every single thread are slow. I get a max of 50MBps (400mbps) per thread (which drops to 30MBps (240mbps) when processing a very large VM). Why can I higher speed when running multiple threads but 1 thread is stuck at a max of 400mbps. Storage is barely using and IOMeter gives me much better performance using 100% random read (125MBps (1gbps)) on a single thread.
-
- Influencer
- Posts: 10
- Liked: never
- Joined: May 26, 2014 4:26 pm
- Full Name: Michel Deschenes
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
Yeah I figured that out when checking the network usage, it had the VEEAM server as the source and destination, as if it was using the loopback to communicate with the agents. It did change to source as the bottleneck later on.v.Eremin wrote: Even when executed on the same machine, VB&R source and target components communicate with each other via TCP. The traffic stays local, though. This communication has been identified as the bottleneck in your case. Thanks.
-
- VP, Product Management
- Posts: 27377
- Liked: 2802 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: How to confirm VEEAM v7 is using DirectSan?
Do you have any virtual proxy server to compare job performance of HotAdd vs. SAN backup modes? BTW, do you read and write data to the same SAN storage at the same time?
Who is online
Users browsing this forum: Google [Bot], m.costantino and 45 guests