Comprehensive data protection for all workloads
Post Reply
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Storage integration, what are the benefits?

Post by HendersonD »

We installed Veeam 9 about five months ago. We use Nimble Storage so we upgraded to Veeam 9.5 when it was released since it has Nimble integration. I have done a bunch of reading and am still foggy on exactly what are the advantage(s) when storage integration is enabled. I do know if storage integration is enabled:
  • There is a storage snapshot taken at the beginning of a backup/replication job and released at the end of the job
  • My backup/replication jobs take a bit longer to run, not sure if this is normal
Past that, I am not sure what the advantage(s) are
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Storage integration, what are the benefits?

Post by PTide »

Hi,

Actually backup from storage snapshots is supposed to speed up backup and replication operations and reduce the impact of VMware vSphere snapshot removal on the production environment. Could you please provide some numbers to compare job durations w/ and w/o storage snapshot?

Thanks
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Storage integration, what are the benefits?

Post by foggy »

HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Re: Storage integration, what are the benefits?

Post by HendersonD »

So far I have only enabled storage integration with my replication job. We replicate 15 VMs from my Nimble array in production to my Nimble array in my DR site. Approximately 14 minutes without replication, about 16 minutes with.

Yes, everything is setup per link you provided. Is there anyway to test whether the replication job is using the storage snapshot while it is running? I do see the step of "Creating storage snapshot" at the beginning of job and releasing them at the end of the job. Is that enough to tell me that it is truly replicating from a storage snapshot?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Storage integration, what are the benefits?

Post by foggy »

You should see something like "Using backup proxy <proxy name> for retrieving Hard Disk 1 data from storage snapshot" in the job session log.
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Re: Storage integration, what are the benefits?

Post by HendersonD »

I am seeing "Using backup proxy <proxy name> for retrieving Hard Disk 1 data from storage snapshot" for each of the VMs being replicated. So my replication job is using storage snapshots. So back to my original question, what is the advantage of using storage integration?

Right now my replication job is running a bit slower with storage integration when it should be running faster. What is causing this?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Storage integration, what are the benefits?

Post by foggy »

What are the bottleneck stats for both jobs? Data flow is a bit different, that might be the reason.
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Re: Storage integration, what are the benefits?

Post by HendersonD »

I will have to run the job both ways and look carefully at the bottlenecks. I know when storage integration is enabled the storage snapshot at the beginning of the job happens very quickly. Once all VMs are replicated, from that point it takes about 2 to 3 minutes for the storage snapshots to be deleted. There are 10 luns being snapshotted. I thought this step, which is an API call from Veeam to Nimble, would happen in seconds. I am thinking this is where the extra time comes from.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Storage integration, what are the benefits?

Post by foggy »

Let's wait for the full bottleneck stats prior to making any conclusions.
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Re: Storage integration, what are the benefits?

Post by HendersonD »

One thing that also may have a bearing on this is my Nimble Storage. I am using one of their all flash arrays in production so it has very high IOPS and low latency. Would replication from storage snapshots versus replication from live data really show much difference in performance in this situation?
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Re: Storage integration, what are the benefits?

Post by HendersonD »

I ran a replication job just to get everything as up to date as possible since my replication jobs run once an hour
As soon as this job finished, I ran two more replication jobs, one with and one without integration. Here is the length of time each job took along with bottleneck information.

Without storage integration 13:02
Load: Source 83% > Proxy 12% > Network 53% > Target 56%

With storage integration, 17:11
Load: Source 37% > Proxy 19% > Network 56% > Target 72%

There are 5 distinct sections to a replication job
  1. Start of the job - this only takes a few seconds
  2. Create storage snapshots (only if integration is enabled)
  3. The actual replication of the VMs
  4. Removal of storage snapshots (only if integration is enabled)
  5. Removal of oldest restore point to meet retention requirements
When I carefully watch both jobs, it appears that steps 1, 3, and 5 take about the same amount if time. This means that storage integration is just adding on more time with taking and deleting storage snapshots.
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Re: Storage integration, what are the benefits?

Post by HendersonD »

Looking at the bottleneck stats, it appears that the benefit with storage integration is a lighter load on the source array during replica/backup

If integration is turned off and the extra load placed on the source array has little or no impact (VMs still perform fine) then there is no advantage to storage integration. Is this correct?
chrisdearden
Veteran
Posts: 1531
Liked: 226 times
Joined: Jul 21, 2010 9:47 am
Full Name: Chris Dearden
Contact:

Re: Storage integration, what are the benefits?

Post by chrisdearden » 2 people like this post

its more about the impact of VMware snapshot consolidation - without backup from storage snapshots , the VMware snap is open for the whole length of the replication cycle. With a high I/O workload , that could lead to a pretty sizeable snap, which could cause problems when consolidating. Having a fast All-Flash array would mitigate some of this in the first instance , however with Backup from storage snapshots , the time that the VMware snap is open reduces considerably.

That said, if you have a large number of machines with a very low change rate , you may not necessarily see the benefit as much. Because of the additional steps of processing the storage snapshot once all of the VMware snaps have been taken , then removing those snaps , it may be faster in certain circumstances to not use the integration.
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Re: Storage integration, what are the benefits?

Post by HendersonD »

Never thought about the consolidation piece, great point. That alone has me leaning towards enabling it even though it takes slightly longer. I have had VMs that needed snapshot consolidation that I have struggled with getting the consolidation to work well.
itfnb
Enthusiast
Posts: 58
Liked: 9 times
Joined: Mar 12, 2012 8:18 pm
Contact:

Re: Storage integration, what are the benefits?

Post by itfnb » 1 person likes this post

I've seen similar results with storage integration on 3PAR production array...the vast majority of our VMs are on all-flash, too. Since our nightly local backup job takes around 15 min, I opted to skip the integration b/c the "regular" VM snapshot removal takes only a few seconds for each VM. I might revisit the decision since it's been a year or so since that config.

However, for our nightly replication where we're using about 50Mbps to our DR site, the jobs take much longer (several hours) and the storage-integrated snapshots made a HUGE reduction in our runtimes b/c you don't have to wait for long vSphere snapshot removal times.
Post Reply

Who is online

Users browsing this forum: dbaages, ravi1988 and 167 guests