Comprehensive data protection for all workloads
Post Reply
lobo519
Veteran
Posts: 315
Liked: 38 times
Joined: Sep 29, 2010 3:37 pm
Contact:

6.1 Patch 1 = HUGE Improvement!

Post by lobo519 » 2 people like this post

WOW! My backup jobs run time have been cut in half (Reverse Incremental) with patch 1! Craziness! :mrgreen:
davidb1234
Expert
Posts: 162
Liked: 15 times
Joined: Nov 15, 2011 8:47 pm
Full Name: David Borden
Contact:

Re: 6.1 Patch 1 = HUGE Improvment!

Post by davidb1234 »

whats your setup like? we use reverse incremental too and i just installed the patches today. we'll see what happens. all FC SAN, direct attached here.
lobo519
Veteran
Posts: 315
Liked: 38 times
Joined: Sep 29, 2010 3:37 pm
Contact:

Re: 6.1 Patch 1 = HUGE Improvment!

Post by lobo519 »

All my storage is iSCSI. I have a job that backups locally and a second job that backs up to a co-location. Both literally are finishing in half the time.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: 6.1 Patch 1 = HUGE Improvment!

Post by foggy »

lobo519 wrote:WOW! My backup jobs run time have been cut in half (Reverse Incremental) with patch 1! Craziness! :mrgreen:
Thanks for sharing this! I know, our R&D did work really hard to gain this improvement.
homerjnick
Expert
Posts: 212
Liked: 35 times
Joined: Feb 20, 2012 4:13 pm
Full Name: Nick Mahlitz
Contact:

Re: 6.1 Patch 1 = HUGE Improvment!

Post by homerjnick »

Hmmm installed the patch and everything is fine but haven't had time to check timings...good point I will do...
pendragon8062
Lurker
Posts: 2
Liked: 1 time
Joined: May 10, 2011 4:11 pm
Full Name: Chris
Contact:

Re: 6.1 Patch 1 = HUGE Improvment!

Post by pendragon8062 » 1 person likes this post

I must concur. Our synthetic full and transform (previously a 22hr job on 6.1) took 10 hrs after applying patch 1.

Many thanks to the Veeam team!

Chris
chimera
Enthusiast
Posts: 57
Liked: 3 times
Joined: Apr 09, 2009 1:00 am
Full Name: J I
Contact:

Re: 6.1 Patch 1 = HUGE Improvement!

Post by chimera »

question is, what caused the supposed poor performance in the first place? bug fix or utilisation of an "undocumented vmware feature?" 8) after patch 1, our backups are more stable than before, most vm's backup performance is no different (10GbE iscsi, direct san, reversed incrementals) but i do see that some vm's are significantly quicker (2.5 times faster). I assume the variation is due to the amount of data changed (or not) on each vm.

We do Veeam backups to local storage ( 6 x 2TB SATA's in RAID 0 ) then off to tape afterwards. I see Veeam is noting that in half the jobs, the target is the bottleneck, so am going to upgrade write cache in the RAID controller see how much improvement this makes.
VER
Influencer
Posts: 23
Liked: 4 times
Joined: Jan 16, 2011 10:24 am
Full Name: Wouter
Contact:

Re: 6.1 Patch 1 = HUGE Improvement!

Post by VER » 1 person likes this post

Same here. I went from a duration of almost 8 hours to a duration of 3:15. :D
I'm very happy with this update.

Our physical backup server has iscsi connections to the SAN, backups are stores on local disks.
Gostev
Chief Product Officer
Posts: 31456
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 6.1 Patch 1 = HUGE Improvement!

Post by Gostev » 4 people like this post

chimera wrote:question is, what caused the supposed poor performance in the first place? bug fix or utilisation of an "undocumented vmware feature?"
As you may already know - under the hood, our backup files have pretty complex architecture. They implement single instancing for block-level deduplication, B-tree indexes for high-performance block lookup (important for instant restores), as well as transactional updates (to ensure data is not lost due to crash or power off) implemented through some data redundancy.

Transactional updates require knowing for sure that certain data is committed to the storage. This can only be done by issuing low-level flush command to the storage device. Previously, we would flush data too often - and our engine was spending most of the time waiting for flushes to complete. Now, flush happens more rarely, which in turn improves performance significantly. Updates to the backup storage are still transactional, just happen with bigger chunks of data. The only drawback is very slight (2GB on average) increase of VBK size (more space is required to maintain data redundancy with more rare update cycles).

One other performance enhancement relates to using direct SAN access mode in large deployments with many LUNs, this one is best classified as optimization. Originally, we simply did not realize certain operations can take so much time in large environments, and did not pay much attention to optimizing certain parts of our code.

Version 6.5 will have additional enhancements around direct SAN access that we have recently identified thanks to this post and willingness of this customer to work closely with us. I can already say that in 6.5, you can expect a few more minutes cut down from each virtual disk processed in direct SAN access mode, resulting in significant overall backup performance improvement. We never stop improving ;)
Gostev
Chief Product Officer
Posts: 31456
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 6.1 Patch 1 = HUGE Improvement!

Post by Gostev »

chimera wrote:i do see that some vm's are significantly quicker (2.5 times faster). I assume the variation is due to the amount of data changed (or not) on each vm
Correct.
deduplicat3d
Expert
Posts: 114
Liked: 12 times
Joined: Nov 04, 2011 8:21 pm
Full Name: Corey
Contact:

Re: 6.1 Patch 1 = HUGE Improvement!

Post by deduplicat3d »

Thanks for the technical details... Got my CS degree at Cal, so I appreciate the small things.
babyjuale
Lurker
Posts: 1
Liked: never
Joined: Dec 31, 2010 5:58 pm
Full Name: Andrew Edwards
Contact:

Re: 6.1 Patch 1 = HUGE Improvement!

Post by babyjuale »

Anyone here with synthetic backups configured in their jobs and can speak to the improvements they are seeing with Patch 1?


Thanks.
Starman
Enthusiast
Posts: 44
Liked: 10 times
Joined: Sep 27, 2011 5:11 pm
Full Name: Todd Leavitt
Contact:

Re: 6.1 Patch 1 = HUGE Improvement!

Post by Starman » 3 people like this post

My backups are also. my Veeam vm has 8 cpu's assigned to it and now vCenter keeps flinging me warnings about CPU utilization. Nothing wrong with that but its clearly hitting CPU a lot harder!

The VMworld Veeam party was great! Saw Gostev in the Veeam session but seeing him get down on the dance floor with his girl friend / wife to techno was priceless!!
Gostev
Chief Product Officer
Posts: 31456
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 6.1 Patch 1 = HUGE Improvement!

Post by Gostev » 2 people like this post

Hmm, actually she is former Veeam employee, and she was with her husband at the party (who is also my friend) :D but yes, she is the best dancer I've ever met. Wow, this is worst offtopic ever, haha!
deduplicat3d
Expert
Posts: 114
Liked: 12 times
Joined: Nov 04, 2011 8:21 pm
Full Name: Corey
Contact:

Re: 6.1 Patch 1 = HUGE Improvement!

Post by deduplicat3d »

hahaha, I missed more than just vSphere!
zoltank
Expert
Posts: 229
Liked: 41 times
Joined: Feb 18, 2011 5:01 pm
Contact:

Re: 6.1 Patch 1 = HUGE Improvement!

Post by zoltank »

Gostev wrote:Hmm, actually she is former Veeam employee, and she was with her husband at the party (who is also my friend) :D but yes, she is the best dancer I've ever met. Wow, this is worst offtopic ever, haha!
I nominate this post for Post Of The Year.
Post Reply

Who is online

Users browsing this forum: No registered users and 177 guests