Comprehensive data protection for all workloads
Post Reply
BarryH
Novice
Posts: 3
Liked: never
Joined: Jul 21, 2013 10:14 pm
Full Name: Barry Hohstadt
Contact:

Backing up large file shares

Post by BarryH »

We have a file server that has about 10 TB of data on one drive. I have removed that drive from the Veeam backup and I am backing up the data with a different system. I was under the impression that it was not a good idea to snapshot that large of a disk, but it would be nice to use Veeam for all of my backups. What is the best practice for backing up that much file data?
Thanks
DaveWatkins
Veteran
Posts: 370
Liked: 97 times
Joined: Dec 13, 2015 11:33 pm
Contact:

Re: Backing up large file shares

Post by DaveWatkins »

I don't have anything that big but I don't see why it would be a problem. The major concern I would expect is filling up the LUN it's hosted on if the change rate is high, but given it's a file server that's probably fairly unlikely. If the change rate is fairly low the snapshots shouldn't be around that long, the hardest part will be the initial full backup and the time it takes to do that

I assume you can't break it down into smaller drives by splitting the data up?
csinetops
Expert
Posts: 113
Liked: 16 times
Joined: Jun 06, 2014 2:45 pm
Full Name: csinetops
Contact:

Re: Backing up large file shares

Post by csinetops »

I have a couple large file servers with multiple large disks around 4TB each. While they aren't 10TB like yours , Veeam doesn't seem to mind bigger disks. One thing to keep in mind is recovery speed, the bigger the disk(s) the longer it's going to take to recover.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backing up large file shares

Post by foggy »

VM of that size shouldn't be a problem provided you're meeting the backup window. Please see another thread regarding similar matter.
nunciate
Expert
Posts: 247
Liked: 39 times
Joined: May 21, 2013 9:08 pm
Full Name: Alan Wells
Contact:

Re: Backing up large file shares

Post by nunciate »

I have several file servers that I backup using reverse incremental with Veeam. One of the servers has 5x 2Tb drives so a total of 10Tb usable. Windows 2012 Dedup is turned on for all these file servers so on this big one I am maybe using 5Tb of that space. The other servers are pretty large as well. I backup with no issues. If you are using Windows 2012 servers for your repositories are are using 2012 dedup on those repositories it is a good idea to exclude the folder that contains your vbk file as it can get corrupted when deduping a file that large.
BarryH
Novice
Posts: 3
Liked: never
Joined: Jul 21, 2013 10:14 pm
Full Name: Barry Hohstadt
Contact:

Re: Backing up large file shares

Post by BarryH »

Thanks everyone, great information. Hopefully I can archive off some of that data before using Veeam. I'll be happy to have all of my backups under one roof!
Delo123
Veteran
Posts: 361
Liked: 109 times
Joined: Dec 28, 2012 5:20 pm
Full Name: Guido Meijers
Contact:

Re: Backing up large file shares

Post by Delo123 »

Hi Barry,

we have multiple 10TB+ Fileservers (even with Windows Dedupe enabled so actual data is about 3-5x that much). Due tue Dedupe changerate is quite high but still we have no issues at all backing them up with veeam since i can remember.
Restoring the VM in an emergency is certainly not the easiest (quickest), but that is what replica's are for :) If your underlying Storage / Repository is performant there should be no issues, but since it seems like you are doing some kind of agent backup and that's working Veeam should be your littlest problem. Have all components up to date always makes a big difference. ReFS and upcoming vmfs-6 when veeam support is released should be a great combination :)
smd32
Service Provider
Posts: 14
Liked: never
Joined: Jan 24, 2016 4:34 am
Full Name: Scott Drassinower
Contact:

Re: Backing up large file shares

Post by smd32 »

I haven't had >5TB for a VMDK backup, but these have been working reliably for a couple of years now up to that size. Lots of 1TB Exchange volumes that even on a busy server, can get backed up successfully.

What I could not get to work with a 15TB volume (about 10TB of data) was VEB. It would BSOD the server running VEB before getting to half completed. This is an iSCSI disk on a 2012 cluster for file shares, not VMDK. We went back to using the original product from a different vendor to back things up, but I'm hoping the upcoming Veeam Agent for Windows will be able to handle this.
ejfarrar
Novice
Posts: 6
Liked: 3 times
Joined: Jun 06, 2016 6:12 pm
Full Name: Eric J. Farrar
Contact:

Re: Backing up large file shares

Post by ejfarrar » 1 person likes this post

I have MANY VMs with over 10TB drives - most of my NFS servers use 8TB volumes, but I have one which uses 41x 16TB vmdk's

All of my VMs back up daily. Granted, the larger VMs (60+TB) take about 1 day to do full backups using storage integration - if using NBD and I get good speeds for the throughput, it usually takes about 2-3 days. Daily incremental backups using CBT take about 10-30 minutes for these big ones. The only potential failure point I encounter with any frequency (one big VM every few months) is with vmdk consolidation issues (similar to VMware's phantom snapshot issue).

Enjoy!
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Backing up large file shares

Post by Mike Resseler »

Scott,

Did you logged a support case for that problem with VEB? We have customers that use VEB to backup those data sizes without any issues...

Mike
FSSC
Novice
Posts: 3
Liked: never
Joined: Dec 01, 2015 12:47 am
Contact:

Re: Backing up large file shares

Post by FSSC »

After seeing the sizes of some of the backups here, i'm now convinced something isn't right in our set-up. Our larger backup jobs are 1tb and 2tb or so. On avg, it takes 9 and 12 hours respectively for each incremental job to complete. I see that roughly half the data for each job is being read with actual data transfer amounts of 2.3gb and 12gb being made (not much considering)... I'm going to have my admin open a ticket, but what if anything can i have him check first?

Thanks
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backing up large file shares

Post by foggy »

Bottleneck stats.
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Backing up large file shares

Post by Mike Resseler »

FSSC,

It might be a bit more difficult to get information from VEB compared to what VBR is giving :-)

Let your admin check this: https://helpcenter.veeam.com/endpoint/1 ... urces.html and see if the throttle is active (might be a busy system and therefore the backup goes to the background)

But because most information can be found in the logs, it is advised to create a support case (through the VEB UI) so they can deep-dive into it. Please post your Case ID here and the results of the investigation. (But don't forget: Support call is FREE, but comes with no SLA, on a best effort basis)

Thanks
FSSC
Novice
Posts: 3
Liked: never
Joined: Dec 01, 2015 12:47 am
Contact:

Re: Backing up large file shares

Post by FSSC »

foggy wrote:Bottleneck stats.
Load: Source 98% > Proxy 54% > Network 1% > Target 0%
Duration: 12:49:21 - Processing Rate: 28 MB/s - Processed 1.8 TB / Read: 1 TB /Transferred: 5.8 GB
FSSC
Novice
Posts: 3
Liked: never
Joined: Dec 01, 2015 12:47 am
Contact:

Re: Backing up large file shares

Post by FSSC »

Mike Resseler wrote:It might be a bit more difficult to get information from VEB compared to what VBR is giving :-)
Hi Mike and thanks for the reply. I should clarify that my question relates to VBR not VEB. I have no issues at all with VEB and wish we'd see the same BU performance from VBR that we do from VEB. I guess i'll have my team get a case open.

Thanks again
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Backing up large file shares

Post by tsightler »

If you are reading lots of data, but transferring very little, this usually indicates one of two things, something wrong with CBT, or something in-guest is touching lots of blocks, making VMware believe the blocks have been changed but, for whatever reason, there's no actual changed data. I'd suggest starting with a CBT reset, verify that CBT is actually working (i.e. no surprise HWv4 VMs), and work from there.
ddrrdd
Novice
Posts: 6
Liked: never
Joined: Oct 13, 2016 4:01 pm
Contact:

Re: Backing up large file shares

Post by ddrrdd »

10 To on only one disk can be huge for you (depend of the performance of your hardware / storage bay)

If all your data of that server is only on one drive, I kindly recommend you to split that drive in multiple smaller disk/vmdk an aggregate them in one drive/mount point using the OS disk manager / lvm.

I administer many DB and you should expect better performance with multiple stream to read/write your data than only one for your 10 To.

You should expect better virtualisation performance in many points: you can use multiple LUN / datastore in your storage, multiple virtual HBA in vmware, your backup can read multiple disk in parallel (restore will also be faster), your snapshot commit will be more efficient…

In linux lvm usage is very common, no so with Windows but there is also a performant disk manager. Pay attention to disk resizing if you plan to grow your volume and use the stripping aggregation (test the resizing need with small virtual disks before your 10 To)

Best Regard,
David
Seve CH
Enthusiast
Posts: 67
Liked: 29 times
Joined: May 09, 2016 2:34 pm
Full Name: JM Severino
Location: Switzerland
Contact:

Re: Backing up large file shares

Post by Seve CH »

Hello.

I am against too many disks. In my experience, the more disks the VM has, the longer the stunt time is while taking or removing snapshots. That could lead to time-outs (and cluster failovers), sockets closing and so on.

For instance, we gave up with frequent incrementals or replications in our file servers (7-8 vDisk, 7-8TB) because the stunt time was 6 to 8 seconds and every snapshot operation (especially taking snapshot) generated 2000-4000 handles closed by clients and Citrix problems with profiles being inaccessible. Now we make only daily backups at night of these servers (way far from the RPO of 15 minutes for all application Veeam announces though).

We have an image archive (PACS) with the same problem wich must be backuped frequently and what we have done is to consolidate data in less disks via extend disk, add mirroring of the old partition and then remove the old disk mirror. With that we removed 10 vDisks and the stunt time of that server (8TB) went down to approx. 2 seconds and the application doesn't crash anymore (8-10 seconds stunt was too much). We didn't consolidate more because vmware doesn't allow to do hot-extend over 2TB.

In all cases the problem was not the size of the VM, but the number of disks and marginally the load on the VM and host. Tested on ESX 5.1, 5.5, 6.0. I have no idea if the stunt times are better in 6.5. It is not a Veeam problem, but a vmware "feature" (reproductible while taking manual snapshots from vSphere client of test VMs).

Best regards.
mkreitzer
Novice
Posts: 8
Liked: 1 time
Joined: Dec 17, 2015 3:54 pm
Full Name: Michael Kreitzer
Contact:

Re: Backing up large file shares

Post by mkreitzer »

Just don't expect to be backing them up to tape. Even though 9.5 finally made it possible to avoid having tape copy jobs terminate when the primary backup job starts, I don't think Veeam is capable of parallel tape drive usage when copying a single VM even in 9.5. We backup multiple 10TB+ VM's daily, but a single LTO6 drive takes days to copy a full backup of just one of them.

It's extraordinarily aggravating.

Also, Seve CH, you're adorable. :) It's "stun time", btw. It's a shame that with the huge head start VMware had that they could not implement a ReFS 3.0/NetApp/Everybody but VMware style snapshot system before Microsoft. We still have to consider snapshot deletes a potentially disruptive event.
Delo123
Veteran
Posts: 361
Liked: 109 times
Joined: Dec 28, 2012 5:20 pm
Full Name: Guido Meijers
Contact:

Re: Backing up large file shares

Post by Delo123 »

A SAN with Veeam Snapshot integration would be nice for these cases, but i stopped trying figuring out how Veeam decided who to integrate. Hp/EMC and then of all possible brands Veeam choose Nimble? Anyway, a lot of Vendors tell me Veeam has no resources and they are all ready blabla, time will tell i guess :(
Post Reply

Who is online

Users browsing this forum: bct44, Google [Bot] and 187 guests