Comprehensive data protection for all workloads
iroche
Enthusiast
Posts: 30
Liked: never
Joined: May 20, 2011 8:27 am
Full Name: Ian Roche
Location: Ireland
Contact:

What size is the largest vm you backup ?

Post by iroche »

Just a quick Q out of interest more than anything . The biggest vm I am backing up is our corp file server coming in at just over 7TB spread across 5 VMDKS (breakdown below). Whats the biggest virtual machine you are backing up and have you ever tried to restore it??? I haven't tried the instant restore of this virtual machine as of yet. I keep 8 restore points on disk and run a incremental with a Synthetic job running every Saturday which then kicks off a tape backup of the full vbk. The backup with all the restore points consumes 5.2TB of space on the backup server.

System VMDK - 60GB
Disk 1 VMDK - 1.9TB
Disk 2 VMDK - 1.9TB
Disk 3 VMDK - 1.9TB
Disk 4 VMDK - 1.9TB
Disk 5 VMDK - 1.1TB

Cheers
Ian
mvoinchet
Lurker
Posts: 2
Liked: never
Joined: Sep 21, 2011 7:03 pm
Full Name: Michael Voinchet
Contact:

Re: What size is the largest vm you backup ?

Post by mvoinchet »

Hi iroche,

We are in the process of virtualizing our central project file server which has approx. 8TB of project data on it that is connected via a RDM and our storage is on Dell Equallogic arrays.. I apologize if this question is a bit off topic, but how did you accomplish the spanning between your vmdk files? Did you use Windows Disk spanning? Our plan is to migrate the project data to another equallogic array that hosts the vmdk files and are spanned in Windows to achieve the neccessary drive space to accomodate the data.

Overall we are pleased with Veeam as we currently backup our Exchange 2010 infrastructure (Only about 600GB) with it as well as many other smaller servers.

Thanks,
Mike
iroche
Enthusiast
Posts: 30
Liked: never
Joined: May 20, 2011 8:27 am
Full Name: Ian Roche
Location: Ireland
Contact:

Re: What size is the largest vm you backup ?

Post by iroche »

I created one LUN per VMDK . So as above I have 5 LUNS and each one I created a VMFS volume on each of these live each VMDK which are mapped to the file server.


Hope this helps.
Ian
Patje
Novice
Posts: 7
Liked: never
Joined: Jul 08, 2011 11:20 am
Full Name: Patrick van Beek
Contact:

Re: What size is the largest vm you backup ?

Post by Patje »

mvoinchet wrote:Hi iroche,

We are in the process of virtualizing our central project file server which has approx. 8TB of project data on it that is connected via a RDM and our storage is on Dell Equallogic arrays.. I apologize if this question is a bit off topic, but how did you accomplish the spanning between your vmdk files? Did you use Windows Disk spanning? Our plan is to migrate the project data to another equallogic array that hosts the vmdk files and are spanned in Windows to achieve the neccessary drive space to accomodate the data.

Overall we are pleased with Veeam as we currently backup our Exchange 2010 infrastructure (Only about 600GB) with it as well as many other smaller servers.

Thanks,
Mike
Mike, I would go with Ian's setup and just use different drive letters instead of creating one big disk with disk spanning.
If I recall correctly, you need to create dynamic disks in order to stripe them, and dynamic disks and equallogic don't really get along.
Suppose it should work, and your volume gets corrupt for some reason, you'll be restoring quite a lot of data, no matter how good Veeam is.
So different disks, and use DFS for instance to present a unified share.

Hope it helps :)
Patrick
mvoinchet
Lurker
Posts: 2
Liked: never
Joined: Sep 21, 2011 7:03 pm
Full Name: Michael Voinchet
Contact:

Re: What size is the largest vm you backup ?

Post by mvoinchet »

Thanks Ian and Patrick for the prompt replies,

Our main project file server has a central drive letter mapped for everyone so creating separate drive letters for each VMFS (VMDK) volume would not work for us. I will do some further research regarding DFS however. Our technology vendor made the windows disk spanning feature seem like it wasn't a big deal to do, but I have to admit that I've seen several warnings about it in other places. We have run a data usage assessment in regards to archiving the older data that isn't being touched to reduce the size, but we still have move than 2TB of active project data, so we will still be exceeding the vmdk limts.

Thanks again guys.

Mike
Reimold
Enthusiast
Posts: 41
Liked: 1 time
Joined: Sep 07, 2009 11:58 am
Full Name: Dirk Reimold
Contact:

Re: What size is the largest vm you backup ?

Post by Reimold »

iroche wrote: System VMDK - 60GB
Disk 1 VMDK - 1.9TB
Disk 2 VMDK - 1.9TB
Disk 3 VMDK - 1.9TB
Disk 4 VMDK - 1.9TB
Disk 5 VMDK - 1.1TB
Hi Ian,

i have nearly the same sized file server that i backup with veeam v6. I run the job in "reversed incremental" mode and just want to know how long does your job run when it creates a "synthetic full" ?

br

Dirk
iroche
Enthusiast
Posts: 30
Liked: never
Joined: May 20, 2011 8:27 am
Full Name: Ian Roche
Location: Ireland
Contact:

Re: What size is the largest vm you backup ?

Post by iroche »

It takes around 8 hours for the Synthetic full to complete, I then have a scripted BE job that runs and send the VBK to tape ( 10 LTO4 tapes in total).
VirtualPlaya
Novice
Posts: 3
Liked: never
Joined: Mar 02, 2010 5:18 pm
Full Name: Vitual Playa
Contact:

Re: What size is the largest vm you backup ?

Post by VirtualPlaya »

iroche wrote:It takes around 8 hours for the Synthetic full to complete, I then have a scripted BE job that runs and send the VBK to tape ( 10 LTO4 tapes in total).
Only 8 hours for a full synthentic of around 9 TB???? Wow!!!
Sebi!
Novice
Posts: 9
Liked: never
Joined: Jan 04, 2011 9:30 am
Contact:

Re: What size is the largest vm you backup ?

Post by Sebi! »

That´s really fast. So does someone know what the main points are for a fast synthetic full? When i check my stats i think its alot the CPU performance.

The duration at my system for a file server increased very much.

1.2TB -> 9h
1.6TB -> 21h
2.1TB -> 47h

And now i wanna do something that i get a "normal" time back.

Thanks
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: What size is the largest vm you backup ?

Post by foggy »

Target storage IOPS are much more important. Essentially, synthetic full is nothing but I/O load on the storage (as it produces random reads/writes versus sequential writes in case of full backup).
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: What size is the largest vm you backup ?

Post by Gostev »

VirtualPlaya wrote:Only 8 hours for a full synthentic of around 9 TB???? Wow!!!
That would require about 300MB/s throughput. Fast storage there, for sure!
NightBird
Expert
Posts: 242
Liked: 57 times
Joined: Apr 28, 2009 8:33 am
Location: Strasbourg, FRANCE
Contact:

Re: What size is the largest vm you backup ?

Post by NightBird »

Humm... 300MB/S throughtput on random io patern !!! (as synthetic full do heavy random read/writes), I'm doubtful.
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: What size is the largest vm you backup ?

Post by Gostev »

Actually, synthetic full is about 90% sequential reads (aside of changed blocks from incremental backup files), so there is a lot of read ahead which most SANs are optimized for. And there are 100% sequential writes into the new full. That is, aside of randomness from the fact that both reads and writes are going in parallel.

It is transform option which adds heavy random I/O to synthetic fulls.
vcharlie
Veeam Software
Posts: 218
Liked: 45 times
Joined: Jan 07, 2011 2:43 am
Full Name: Charles Clarke
Contact:

Re: What size is the largest vm you backup ?

Post by vcharlie »

On the topic of Instant Recovery, we have a customer in Asia that tested Instant VM Recovery on a VM that was around 7TB in size. The VM came back in under 3 minutes. Nice!
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: What size is the largest vm you backup ?

Post by Gostev »

VM size does not affect its boot speed, of course. However, VM size does affect the speed of "normal" VM restore that you would have to do with other backup solution. It would probably take a few days with 7TB of data ;)
Sebi!
Novice
Posts: 9
Liked: never
Joined: Jan 04, 2011 9:30 am
Contact:

Re: What size is the largest vm you backup ?

Post by Sebi! »

foggy wrote:Target storage IOPS are much more important. Essentially, synthetic full is nothing but I/O load on the storage (as it produces random reads/writes versus sequential writes in case of full backup).
Thanks for the reply. Now I checked my system and it´s not the processor but also not my storage. Right now I only have a 32bit Windows 2003 and it´s my RAM.
So I think i have to install an 64bit System.
slam
Novice
Posts: 7
Liked: never
Joined: Dec 18, 2010 12:19 am
Full Name: Stephen
Contact:

VM too big to be backup by Veeam

Post by slam »

[merged]

To be more accurate, I should say VM to big that Veeam can backup but fail to restore.

We have a Windows 2008 R2 file server running on ESXi5 with eight luns of 2TB each, total 16TB of file storage. The server currently has over 8TB of data and it's growing fast. We use Veeam to backup the server, full backup every month and incremental every day. Ever since the .vbk file grew beyond 7TB in size, we started running into problem doing any kind of restore. Both file level restore and instant recovery failed. Even though the backup job is still running successfully every day. Of course we didn't notice the problem until we need to restore files from your backup.

We have a ticket open with Veeam support for over two months now and we are still running in circle with no real solution. They even sent us a custom build Veeam agent file but that didn't fix the problem. We have a Windows 2008 R2 server with 8 CPUs and 8GB RAM running the backup and repository server and backup proxy. A Windows 2008 R2 server with 8 CPU and 12GB RAM with iSCSI attached as the repository server.

Is anyone out there using Veeam to backup AND restore VM with this much data successfully?
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: What size is the largest vm you backup ?

Post by dellock6 »

My personal record was a Zimbra Server (CentOS 5 based) with 9 vmdk for a total of 2 Tb, powered on with Instant Recovery and restored in production via Storage vMotion.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
slam
Novice
Posts: 7
Liked: never
Joined: Dec 18, 2010 12:19 am
Full Name: Stephen
Contact:

Re: What size is the largest vm you backup ?

Post by slam »

Our vbk file is 8.4TB. Backup is still running but we cannot restore anything. Neither Instant Recovery or file level restore works.
slam
Novice
Posts: 7
Liked: never
Joined: Dec 18, 2010 12:19 am
Full Name: Stephen
Contact:

Re: What size is the largest vm you backup ?

Post by slam »

I've just got off the phone with tier-2 support on our case. I was told that Veeam does not recommend backing up VM's barger than 5TB. I wish they'd have told us that when we evaluate the product.
jrickman
Lurker
Posts: 1
Liked: never
Joined: Mar 05, 2012 11:26 am
Full Name: Jonathan Rickman
Contact:

Re: What size is the largest vm you backup ?

Post by jrickman »

8TB here, with a change rate of around 100GB over 24 hours. Full backups and snapshot removal are painful. Wish there was more fine grained job scheduling to control when a snapshot removal takes place for long running jobs like this.
bobbyk18
Novice
Posts: 5
Liked: never
Joined: Feb 09, 2012 3:42 pm
Full Name: Bobby
Contact:

Re: What size is the largest vm you backup ?

Post by bobbyk18 »

10TB allocated
8TB of it used

We separate it into two jobs:
one job for 6TB of it with a 4.3TB VBK run nightly
one job for 4TB of it with a 2.5TB VBK run once a month(archived data that doesn't change)
running ESXi 5 and Veeam 6 with an Equallogic PS6000.

We're now working on replicating it to a second site using Veeam. Our VRBs are around 45GB per day.

Backups are pretty good, replicas are a pain.
sergeadam
Enthusiast
Posts: 38
Liked: 2 times
Joined: Dec 07, 2011 5:21 pm
Full Name: Serge Adam
Contact:

Re: What size is the largest vm you backup ?

Post by sergeadam »

slam wrote:I've just got off the phone with tier-2 support on our case. I was told that Veeam does not recommend backing up VM's barger than 5TB. I wish they'd have told us that when we evaluate the product.
Then I'm in trouble because one of my file servers is at 8TB and growing.

I have it setup with Windows doing spanning of a bunch of 1.9TB VMDKs, all on the same LUN on an Eq PS4000 with no issue. The backups do seem to work.
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: What size is the largest vm you backup ?

Post by dellock6 »

Can someone from Veeam confirm this? And most of all, if is only a "best practice" or a proper limit (I'm suspecting the former...)
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: What size is the largest vm you backup ?

Post by Vitaliy S. »

Hi Guys,

Not sure where this recommendation comes from, but I guess there was a misunderstanding between Stephen and our support engineer, I believe our support engineer might be talking about these 5 TBs (pretty old thread).

Thanks!
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: What size is the largest vm you backup ?

Post by Gostev »

Yes, most likely the engineer simply misunderstood my very old comment that Vitaly had referenced. I know for the fact they pick up lots of knowledge from these forums ;) anyway, this 2 year old comment was made in v4 times, so it is not really applicable with all the engine improvements made since then, including new backup modes and especially new v6 architecture.
slam
Novice
Posts: 7
Liked: never
Joined: Dec 18, 2010 12:19 am
Full Name: Stephen
Contact:

Re: What size is the largest vm you backup ?

Post by slam »

Sergeadam, have you tested restoring any file using file level restore or instant recovery? I am also curious as how file level restore with Windows spanning volume.

As a best practice, we should test restoring the backups regularly. Especially for those of you who have .vbk file size over 7TB to make sure it works. I'd like to hear if other people have similar problem.

I finally tested restoring one of the eight 1.9TB vmdk file successfully last night. Though it took 8 hours to complete. It confirms that the backup works. Now only if Veeam can make the file level restore and Instant Recovery work also.

Gostev/Vitaliy, my support case is ID#5189322. If you can take a look, I'd greatly appreciate.
slam
Novice
Posts: 7
Liked: never
Joined: Dec 18, 2010 12:19 am
Full Name: Stephen
Contact:

Re: What size is the largest vm you backup ?

Post by slam »

I got the confirmation from the Veeam development team that we hit the architectural limitations of the VeeamAgent, which is still 32-bit. They are going to make a custom VeeamAgent that will allow us to specify the VD to mount for File Level Restore.

The problem here is not with backing up but restoring very large VMs >7TB using File Level Restore and Instant Recovery. For those of you who are using Veeam to backup large VMs, > 7TB, I'd suggest you guys to test if File Level Restore and Instant Recovery are working or not.

Gostev, Is that any plan to upgrade the VeeamAgent to 64-bit?
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: What size is the largest vm you backup ?

Post by Gostev »

Frankly, we did not even consider doing before your post above. Somehow, no one else had run into this limitation in the past 5 years, and we just did not see any reason to have 64 bit agent. Now, I need to discuss this with the devs and get their opinion on the effort required. Of course, we have no choice but to do this.
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: What size is the largest vm you backup ?

Post by tsightler »

I've pretty much always suggested that clients with servers this large to split the backup into multiple jobs, excluding specific disks to keep the total size down, but yes, more and more clients have servers that are this large or larger so it would be good to break this limit going forward.
Post Reply

Who is online

Users browsing this forum: Gostev, jp.verlande, RValensise and 159 guests