How sequential are writes to repository?

VMware specific discussions

How sequential are writes to repository?

Veeam Logoby pkelly_sts » Wed Apr 13, 2016 2:24 pm

I'm just curious as to how much load I'm putting on a repository if I increase the number of "Max concurrent tasks" to it, and am sending backups to it from 1 x San attached proxy & 3 x Proxy VMs. Does all the incoming data get written sequentially as a single "stream" of data to disk, or is there any kind of random disk thrashing taking place?

In this instance I'm talking about a single backup job of multiple VMs as opposed to multiple backup jobs.
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: How sequential are writes to repository?

Veeam Logoby foggy » Wed Apr 13, 2016 2:38 pm

I/O pattern depends on the backup method the job is using.
foggy
Veeam Software
 
Posts: 14742
Liked: 1079 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: How sequential are writes to repository?

Veeam Logoby pkelly_sts » Wed Apr 13, 2016 2:47 pm

Perfect, thanks!

(For this particular site an active full is similar speed to a synthetic full anyway (2.1Tb > 900Gb in 3hrs or so) so I've always just done active full weekly so for this particular site then, backups should be pretty much always sequential.)
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: How sequential are writes to repository?

Veeam Logoby foggy » Wed Apr 13, 2016 3:09 pm

Yes, active full is sequential, however, if touching production VMs each time is not desirable and you still have similar performance with synthetic fulls, you may consider switching to them, at least partially (e.g. weekly synthetic, monthly active).
foggy
Veeam Software
 
Posts: 14742
Liked: 1079 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: How sequential are writes to repository?

Veeam Logoby pkelly_sts » Wed Apr 13, 2016 3:15 pm

Yep, understand the consequences but active fulls really aren't an issue for us on this particular site anyway so as they easily fit into backup windows etc. for a long time I've just let run active ever since first deployment a couple of weeks ago.

Ah, just to clarify, this is active weekly only! Incremental daily of course.

Just for giggles I've increased the concurrent disk tasks from 10 to 12 to see if it makes much/any difference. Wouldn't bother me if it didn't, just interested in knowing the real performance ceiling for the storage we have.
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: How sequential are writes to repository?

Veeam Logoby foggy » Wed Apr 13, 2016 4:22 pm

If your storage is ok with multiple I/O streams, you might want to enable per-VM backup chains as well.
foggy
Veeam Software
 
Posts: 14742
Liked: 1079 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: How sequential are writes to repository?

Veeam Logoby pkelly_sts » Thu Apr 14, 2016 10:08 am

That's high on my list of areas to look into when we upgrade to v9 (which should hopefully be approved this afternoon :)

Now you mention it, I was thinking about per-vm chains this morning and wondered if they also actually help with seeding newly added VMs to off-site backup copy jobs? In other words, if I wanted to add a backup of a new 150Gb VM to a backup copy job (over a link slow enough that I wouldn't want to send that 150gb over the wire for the first full) can you "extract" the local copy of the per-vm backup of that VM, ship it over to remote site, and then use it to seed a copy job that is being sent to a scale out repository based at the remote site with per-vm's enabled?

Hopefully my question makes sense! (In this context it's based on my plan to migrate our last few physicals to VMs & how I might best seed a copy of their first backups to the remote site).
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: How sequential are writes to repository?

Veeam Logoby foggy » Thu Apr 14, 2016 10:20 am 1 person likes this post

This procedure should help. You cannot just copy the full backup for that VM from the per-VM backup chain and use it for seeding.
foggy
Veeam Software
 
Posts: 14742
Liked: 1079 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: How sequential are writes to repository?

Veeam Logoby pkelly_sts » Thu Apr 14, 2016 10:28 am

Ah, interesting-looking approach. I'll admit that at first reading I haven't 100% got my head around it but I'll file it away for when I've had more coffee - thanks for the pointer :)
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: How sequential are writes to repository?

Veeam Logoby foggy » Thu Apr 14, 2016 10:31 am 1 person likes this post

Yep, its pretty creative, but it works.
foggy
Veeam Software
 
Posts: 14742
Liked: 1079 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: How sequential are writes to repository?

Veeam Logoby pkelly_sts » Tue May 03, 2016 9:00 am

Hmm, well having switched to A) Scale-out repository & B) Synthetic weekly full for this specific site I'm really not seeing a whole heap of benefit # in this particular case #.

As I've mentioned above, weekly active fulls aren't much of an issue for us hitting the production environment anyway as they completed inside 2.5hrs or so. Having tried switching back to synthetic full, but to a scale-out repository with two simple repositories configured for performance but (perhaps crucially?) NOT yet enabling per-VM backup chains, the same weekly job took about 5:30hrs so over double the time.

The issue for us is having as wide a window as possible for off-site backup copies over a relatively slow link so losing those 3hrs of copy time is more of a problem, to me.

However, perhaps the per-vm chains would have helped? I doubt it as the first active full of those would still only have been going to the same storage anyway.

To clarify, the original repository is the same as I was previously using (so an active full of 2.2Tb compressed to 850Gb is completed in up to 2.5hrs) but added to the scale out, and I've introduced a second set of spindles of equal performance for the incremental repository part of the scale-out.
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: How sequential are writes to repository?

Veeam Logoby pkelly_sts » Tue May 03, 2016 9:11 am

In fact, double-checking again the last active full took 2:19 in total whereas the synthetic full this weekend took 6:06 in total so nearer 3x longer (in my case, obviously everyone's site/setup is different)
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: How sequential are writes to repository?

Veeam Logoby foggy » Tue May 03, 2016 9:13 am

Looks like active fulls is the way to go in your case. Btw, you didn't mention what kind of repositories your extents are? Just Windows servers?
foggy
Veeam Software
 
Posts: 14742
Liked: 1079 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: How sequential are writes to repository?

Veeam Logoby pkelly_sts » Tue May 03, 2016 9:23 am

Agreed, useful to have tested though.

In this case the server is multi-role: Windows 2008 R2 Server, Dual 6-core, 24Gb mem, B&R Server, Proxy, 8Gb Direct-attach FC SAN (to VMFS) & 14 DAS disks, 12 dedicated to backup storage.

I do have some recently added SAN storage with more spindles per LUN (which is primarily off-site repositories for our main site) which I might be able to find some time to test against.

In fact, I did a similar test for those exact off-site copies in that I re-configured them as a remote scale-out repositoriy (similar setup, two sets of spindles) but I'm having trouble analysing the results to actually figure out how long the transforms are taking for that copy job :-/ (I was just scratching my head trying to figure out exactly what question to ask on the forums to figure out what I'm mis-understanding about transform operations! :) )
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: How sequential are writes to repository?

Veeam Logoby foggy » Tue May 03, 2016 10:30 am

This should be visible from the job stats (transformation is started right after all data transfer operations are completed).
foggy
Veeam Software
 
Posts: 14742
Liked: 1079 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Next

Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 13 guests