Standalone backup agents for Linux, Mac, AIX & Solaris workloads on-premises or in the public cloud
Post Reply
JonJR
Enthusiast
Posts: 38
Liked: 17 times
Joined: Mar 21, 2017 11:25 pm
Full Name: Jon Rhoades
Contact:

Agent Filesystem Limit of 218TB

Post by JonJR »

We've just hit the limit of an agent backup with a Filesystem of 218.5TB and we discovered that the Veeam agent has a hard limit of 218 TB - the docs say "Total size of all file systems included in a file-level backup must not exceed 218 TB". We only actually want to backup 30TB or so of the data. We will see if we can shrink the filesystem to 217TB and get away with that.

However we were wondering if this is an arbitrary limit or an actual technical limitation. We anticipate having file systems greater than 218TB in the future that we would want to agent backup.

For reference, it's a RHEL host with BeeGFS file system, which has XFS as the underlying filesystem.
HannesK
Product Manager
Posts: 14287
Liked: 2877 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: Agent Filesystem Limit of 218TB

Post by HannesK » 1 person likes this post

Hello,
Yes, it is a technical limitation.

As an alternative, you could also consider NAS backup. NAS backup has no limitations on size and performance is much better than Veeam Agent for Linux file-based backup: https://helpcenter.veeam.com/docs/backu ... ml?ver=110

Best regards,
Hannes
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Agent Filesystem Limit of 218TB

Post by PTide »

Hi,
We only actually want to backup 30TB or so of the data. We will see if we can shrink the filesystem to 217TB and get away with that.
I am curious what is the application? Is that a swarm of small files, or rather a bunch of bigger files? How many clients do access them?

Thanks!
JonJR
Enthusiast
Posts: 38
Liked: 17 times
Joined: Mar 21, 2017 11:25 pm
Full Name: Jon Rhoades
Contact:

Re: Agent Filesystem Limit of 218TB

Post by JonJR »

HannesK wrote: Apr 27, 2021 7:16 am Hello,
Yes, it is a technical limitation.

As an alternative, you could also consider NAS backup. NAS backup has no limitations on size and performance is much better than Veeam Agent for Linux file-based backup: https://helpcenter.veeam.com/docs/backu ... ml?ver=110

Best regards,
Hannes
Thanks, will investigate that
JonJR
Enthusiast
Posts: 38
Liked: 17 times
Joined: Mar 21, 2017 11:25 pm
Full Name: Jon Rhoades
Contact:

Re: Agent Filesystem Limit of 218TB

Post by JonJR » 1 person likes this post

PTide wrote: Apr 27, 2021 10:48 am I am curious what is the application? Is that a swarm of small files, or rather a bunch of bigger files? How many clients do access them?
Scientific data (mainly sequence data and mammogram x-rays) that we are analysing using GPUs. And yes, lots and lots of small files... At the top level, we have a "backed up" folder and a "scratch" folder, we didn't want to have totally separate file systems as data gets moved between the two folders regularly. BeeGFS does the tiering between an NVMe tier and a spinning rust tier without changing the file path. Unlike a regular SAN's tiering which is generally reactive, our users promote the data to the NVMe tier by running a BeeGFS command before they run their job in SLURM. Pretty cool stuff!
JonJR
Enthusiast
Posts: 38
Liked: 17 times
Joined: Mar 21, 2017 11:25 pm
Full Name: Jon Rhoades
Contact:

Re: Agent Filesystem Limit of 218TB

Post by JonJR »

HannesK wrote: Apr 27, 2021 7:16 am As an alternative, you could also consider NAS backup. NAS backup has no limitations on size and performance is much better than Veeam Agent for Linux file-based backup: https://helpcenter.veeam.com/docs/backu ... ml?ver=110
Unfortunately what I hadn't realised is that NAS backup is licensed by the TB rather than the server, so it's about $5,000 to back up 50TB this way or if I could shrink my filesystem, it would be included in my B&R subscription.
HannesK
Product Manager
Posts: 14287
Liked: 2877 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: Agent Filesystem Limit of 218TB

Post by HannesK »

sure, it always depends on the needs and technical resources you have. If resizing a filesystem and the speed of VAL is fine, then it sounds reasonable of course.

I just gave you an alternative if technical limitations (speed, size) hit you in a way that require a better solution.
Post Reply

Who is online

Users browsing this forum: No registered users and 10 guests