-
- Enthusiast
- Posts: 30
- Liked: 2 times
- Joined: Feb 13, 2013 12:20 pm
- Full Name: Bob Baird
- Contact:
Optimizing Configuration Setup
Hi All,
I'm looking for some recommendations on the best configuration for my environment.
I have 3 x VMware ESXi 5.0 hosts all connected via FC to a IBM DS3500 SAN. I have 14 Datastores created on the SAN.
Between the 3 hosts, I have approx 36 running VM's.
We are moving from Symantec Backup Exec to Veeam B&R.
I have installed a Synology DS1812+ NAS with 8 x 3TB disks. This has been configured as RAID6, so I have 18TB available storage on the NAS. The NAS is connected to the same Gigabit switch as my 3 hosts.
I have created 2 x VM's - one as my backup server (where I have installed the B&R software) and another as a proxy server.
I have created a single job to save 13 of my 36 VM's - the backups are incremental with a synthetic full being done every Saturday.
This has been configured with the following settings:
Deduplication = Enabled
Compression level = Optimal
Storage optimization = LAN target
Transport mode on the proxy = Virtual Appliance
Max Current Tasks = 4 (as I have assigned 8 vCPU's to my proxy server)
My daily backup jobs are taking just over 1:10 hours to complete, with the backup size being on average 8.0GB of data.
I would appreciate some help or advice on optimizing the configuration for my environment before I start moving more of my VM's off the Symantec backup.
Best Regards,
Bob
I'm looking for some recommendations on the best configuration for my environment.
I have 3 x VMware ESXi 5.0 hosts all connected via FC to a IBM DS3500 SAN. I have 14 Datastores created on the SAN.
Between the 3 hosts, I have approx 36 running VM's.
We are moving from Symantec Backup Exec to Veeam B&R.
I have installed a Synology DS1812+ NAS with 8 x 3TB disks. This has been configured as RAID6, so I have 18TB available storage on the NAS. The NAS is connected to the same Gigabit switch as my 3 hosts.
I have created 2 x VM's - one as my backup server (where I have installed the B&R software) and another as a proxy server.
I have created a single job to save 13 of my 36 VM's - the backups are incremental with a synthetic full being done every Saturday.
This has been configured with the following settings:
Deduplication = Enabled
Compression level = Optimal
Storage optimization = LAN target
Transport mode on the proxy = Virtual Appliance
Max Current Tasks = 4 (as I have assigned 8 vCPU's to my proxy server)
My daily backup jobs are taking just over 1:10 hours to complete, with the backup size being on average 8.0GB of data.
I would appreciate some help or advice on optimizing the configuration for my environment before I start moving more of my VM's off the Symantec backup.
Best Regards,
Bob
-
- Product Manager
- Posts: 20380
- Liked: 2291 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Optimizing Configuration Setup
For more thoughtful piece of advice regarding how backup performance can be increased it’s necessary to have bottleneck statistics for these jobs. Can you provide some? Thanks.
-
- Enthusiast
- Posts: 30
- Liked: 2 times
- Joined: Feb 13, 2013 12:20 pm
- Full Name: Bob Baird
- Contact:
Re: Optimizing Configuration Setup
Hi,
From last nights backup, the bottleneck statistics are showing as follows:
Source 97% > Proxy 28% > Network 3% > Target 3%
Thanks,
Bob
From last nights backup, the bottleneck statistics are showing as follows:
Source 97% > Proxy 28% > Network 3% > Target 3%
Thanks,
Bob
-
- Product Manager
- Posts: 20380
- Liked: 2291 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Optimizing Configuration Setup
Should have mentioned earlier, but can you also provide statistics regarding amount of read/processed/transferred data, since they might be helpful in understanding whether or not CBT mechanism was used?
Additionally, have you considered deploying physical proxy and specifying it in Direct SAN mode; might give a better performance rate? Thanks.
Additionally, have you considered deploying physical proxy and specifying it in Direct SAN mode; might give a better performance rate? Thanks.
-
- Enthusiast
- Posts: 30
- Liked: 2 times
- Joined: Feb 13, 2013 12:20 pm
- Full Name: Bob Baird
- Contact:
Re: Optimizing Configuration Setup
Hi,
Also, we were hoping to avoid having to have a physical server acting as the proxy server, but if it were to provide significant performance increase, then it's an option we could look at.
Thanks,
Bob
Also, we were hoping to avoid having to have a physical server acting as the proxy server, but if it were to provide significant performance increase, then it's an option we could look at.
Thanks,
Bob
-
- Enthusiast
- Posts: 30
- Liked: 2 times
- Joined: Feb 13, 2013 12:20 pm
- Full Name: Bob Baird
- Contact:
Re: Optimizing Configuration Setup
Should have mentioned, looking at the detailed statistics of the 13 VM's, each state that CBT was used.
-
- Veeam Software
- Posts: 21133
- Liked: 2140 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Optimizing Configuration Setup
Bob, what transport mode is being effectively used to retrieve VMs data from the source storage? You should see the corresponding tag ([nbd], [hotadd]) right next to the proxy server name selected for processing in the job statistics window if you select the particular VM to the left.
Also, what is the processing rate for the full job run?
Also, what is the processing rate for the full job run?
-
- Enthusiast
- Posts: 30
- Liked: 2 times
- Joined: Feb 13, 2013 12:20 pm
- Full Name: Bob Baird
- Contact:
Re: Optimizing Configuration Setup
Hi Foggy,
Transport mode is showing as [hotadd].
Also, the processing rate is showing as 495MB/s.
Thanks,
Bob
Transport mode is showing as [hotadd].
Also, the processing rate is showing as 495MB/s.
Thanks,
Bob
-
- Product Manager
- Posts: 20380
- Liked: 2291 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Optimizing Configuration Setup
This is for full backup run, not for an incremental one (statistics of which you’ve provided above), right? Thanks.bob.baird.xedo wrote: Also, the processing rate is showing as 495MB/s.
-
- Enthusiast
- Posts: 30
- Liked: 2 times
- Joined: Feb 13, 2013 12:20 pm
- Full Name: Bob Baird
- Contact:
Re: Optimizing Configuration Setup
Hi,
The 495MB/s relates to the Processing Rate for my incremental backup from last night (for all 13 VM's)
My synthetic full backup takes place at 22:00 on a Saturday evening. Not sure how I retrieve the processing rate for this job.
Thanks,
Bob
The 495MB/s relates to the Processing Rate for my incremental backup from last night (for all 13 VM's)
My synthetic full backup takes place at 22:00 on a Saturday evening. Not sure how I retrieve the processing rate for this job.
Thanks,
Bob
-
- Veeam Software
- Posts: 21133
- Liked: 2140 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Optimizing Configuration Setup
You can list back to the full run in the job stats window using the Back arrow key.
-
- Product Manager
- Posts: 20380
- Liked: 2291 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Optimizing Configuration Setup
The question has to do with an active full backup rather than with a synthetic one. So, apart from synthetic full do you use active full, as well? If so, kindly find its last session (as mentioned by Alexander) and see what numbers are there. Otherwise, just find the very first backup session and get required information from there. Thanks.
-
- Enthusiast
- Posts: 30
- Liked: 2 times
- Joined: Feb 13, 2013 12:20 pm
- Full Name: Bob Baird
- Contact:
Re: Optimizing Configuration Setup
Hi,
Never knew you could do that!
The processing rate for my full backup is only showing as 79MB/s.
Thanks.
Never knew you could do that!
The processing rate for my full backup is only showing as 79MB/s.
Thanks.
-
- Veeam Software
- Posts: 21133
- Liked: 2140 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Optimizing Configuration Setup
Pretty fair speed, not sure you could get better. Bottleneck source means that data cannot be provided by your production SAN to ESXi host any faster.
-
- Enthusiast
- Posts: 30
- Liked: 2 times
- Joined: Feb 13, 2013 12:20 pm
- Full Name: Bob Baird
- Contact:
Re: Optimizing Configuration Setup
v.Eremin wrote:The question has to do with an active full backup rather than with a synthetic one. So, apart from synthetic full do you use active full, as well? If so, kindly find its last session (as mentioned by Alexander) and see what numbers are there. Otherwise, just find the very first backup session and get required information from there. Thanks.
Hi,
We only use incremental with synthetic full - no active full backups.
The difficulty in trying to find the very first backup statistics is that I have added VM's to my original job i.e. I started off with just a single VM and have been adding additional VM's to the list, so I don't have a single point where I have a full backup of all 13 VM's.
Thanks.
-
- Enthusiast
- Posts: 30
- Liked: 2 times
- Joined: Feb 13, 2013 12:20 pm
- Full Name: Bob Baird
- Contact:
Re: Optimizing Configuration Setup
foggy wrote:Pretty fair speed, not sure you could get better. Bottleneck source means that data cannot be provided by your production SAN to ESXi host any faster.
So if I had a physical server acting as a proxy, and attached this directly to my SAN, I should notice considerable improvements in performance?
-
- Product Manager
- Posts: 20380
- Liked: 2291 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Optimizing Configuration Setup
I wouldn't be sure about "considerable", but the usage of Direct SAN method might result in increased performance rates; so, it's worth giving a shot.
Thanks.
Thanks.
-
- Veeam Software
- Posts: 21133
- Liked: 2140 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Optimizing Configuration Setup
You could notice some improvement, depending on the configuration (typically direct SAN and hotadd modes provide quite similar performance).
-
- Enthusiast
- Posts: 30
- Liked: 2 times
- Joined: Feb 13, 2013 12:20 pm
- Full Name: Bob Baird
- Contact:
Re: Optimizing Configuration Setup
Okay, I may just leave things the way they are for now then, especially if my stats aren't showing anything glaringly wrong.
I appreciate all your replies.
Best Regards,
Bob
I appreciate all your replies.
Best Regards,
Bob
-
- Veeam Software
- Posts: 21133
- Liked: 2140 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Optimizing Configuration Setup
Yep, there's definitely nothing to worry about, especially if you meet your backup window with the current setup.
-
- Service Provider
- Posts: 181
- Liked: 48 times
- Joined: Sep 03, 2012 5:28 am
- Full Name: Yizhar Hurwitz
- Contact:
Re: Optimizing Configuration Setup
Hi.bob.baird.xedo wrote:
Also, we were hoping to avoid having to have a physical server acting as the proxy server, but if it were to provide significant performance increase, then it's an option we could look at.
Regarding a physical backup server - the point is not only about best backup performance, but also looking to restore scenarios for several cases.
Most ofter you need to restore specific VM or data within (files/email/db etc).
But in worst case you might need to restore the whole system - such as a human/hardware error causing the SAN to be down.
In such case - the main advantage of physical backup server is that it is independent on the production systems (VMware, san fabric, san storage, etc). Allowing you immediate access to backups in such situation.
About backup performance of physical server in SAN mode vs VM in hotadd mode, I assume that it will be similar. However SAN mode also reduces load from esxi hosts (cpu/ram/disk io) and only SAN storage and backup server + repository are working during backup.
You can also consider a hybrid combination of both physical backup server with additional Veeam proxy running on VM in hotadd mode, which allows you flexibility in testing and choosing what best fits for you.
Please also read the following discussion about it:
Community Forums • View topic - Moving Veeam to Physical Server - Options
http://forums.veeam.com/viewtopic.php?f=2&t=17484
Please note that Veeam running as VM is also a great option like you have now, just the main disadvantage is that it depends on the same infrastructure as the production systems and in some situations it might be an obstacle.
Yizhar
-
- Service Provider
- Posts: 181
- Liked: 48 times
- Joined: Sep 03, 2012 5:28 am
- Full Name: Yizhar Hurwitz
- Contact:
Re: Optimizing Configuration Setup
Hi.
I would also suggest that you plan how to divide your VMs into Veeam backup jobs.
In general I'm favoring more jobs with less VMs - vs single job with all of them.
For example if you have 36 VMs, You can setup about 5-10 jobs with 1-8 VMs per job.
If you are running a multitenant hosting system (or multi department) you can group VMs by tenant/department.
You can group and select VMs to backup in several ways, such as:
individual VM selection.
by VMware folders.
by VMware datastores.
Take a look here - regarding planning VM placement in single vs multiple jobs:
Community Forums • View topic - Optimizing backups for dedup
http://forums.veeam.com/viewtopic.php?t=17169
I would also suggest that you plan how to divide your VMs into Veeam backup jobs.
In general I'm favoring more jobs with less VMs - vs single job with all of them.
For example if you have 36 VMs, You can setup about 5-10 jobs with 1-8 VMs per job.
If you are running a multitenant hosting system (or multi department) you can group VMs by tenant/department.
You can group and select VMs to backup in several ways, such as:
individual VM selection.
by VMware folders.
by VMware datastores.
Take a look here - regarding planning VM placement in single vs multiple jobs:
Community Forums • View topic - Optimizing backups for dedup
http://forums.veeam.com/viewtopic.php?t=17169
-
- Enthusiast
- Posts: 30
- Liked: 2 times
- Joined: Feb 13, 2013 12:20 pm
- Full Name: Bob Baird
- Contact:
Re: Optimizing Configuration Setup
Okay, thanks for your input.
That was actually going to be my next question - how best to add the 36 VM's i.e as a single job, or 4 separate jobs (with 9 VM's in each).
As previously mentioned, my proxy has been configured with 8 x vCPU's which allows me to run 4 concurrent tasks.
So, would I have my 4 backup jobs scheduled to start at the same time?
Thanks,
Bob
That was actually going to be my next question - how best to add the 36 VM's i.e as a single job, or 4 separate jobs (with 9 VM's in each).
As previously mentioned, my proxy has been configured with 8 x vCPU's which allows me to run 4 concurrent tasks.
So, would I have my 4 backup jobs scheduled to start at the same time?
Thanks,
Bob
-
- Veeam Software
- Posts: 21133
- Liked: 2140 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Optimizing Configuration Setup
General recommendation is to group similar VMs (i.e. created from the same templates, for example) to get better dedupe rates. You can then schedule jobs to start at the same time (or with short intervals) and they will just queue up and start automatically according to the number of allowed concurrent tasks.
-
- Enthusiast
- Posts: 30
- Liked: 2 times
- Joined: Feb 13, 2013 12:20 pm
- Full Name: Bob Baird
- Contact:
Re: Optimizing Configuration Setup
Okay, thanks Foggy.
Who is online
Users browsing this forum: uk1984 and 28 guests