Discussions specific to tape backups
Post Reply
philipbehrens
Service Provider
Posts: 6
Liked: never
Joined: Jan 11, 2018 5:42 pm
Full Name: Philip Behrens
Contact:

How to handle high number of files?

Post by philipbehrens » Feb 21, 2019 12:17 pm

Hi,

currently I am facing problems with a file to tape backup job.
The Action "Building protected objects list" took nearly infinity until the backup server runs out of RAM and crashes.
Possibly this is because of the high number of files (12 TB - ~12.000.000 Files).
Is there a way to handle this amount?

Gostev
SVP, Product Management
Posts: 24092
Liked: 3280 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: How to handle high number of files?

Post by Gostev » Feb 21, 2019 12:20 pm

High, what version are you using?

philipbehrens
Service Provider
Posts: 6
Liked: never
Joined: Jan 11, 2018 5:42 pm
Full Name: Philip Behrens
Contact:

Re: How to handle high number of files?

Post by philipbehrens » Feb 21, 2019 12:22 pm

9.5.4.2615
Standard Edition

v.eremin
Product Manager
Posts: 16232
Liked: 1325 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: How to handle high number of files?

Post by v.eremin » Feb 21, 2019 1:46 pm

I'm wondering what type of SQL server you're using. Built-in SQL Express or fully blown SQL server. Thanks!

philipbehrens
Service Provider
Posts: 6
Liked: never
Joined: Jan 11, 2018 5:42 pm
Full Name: Philip Behrens
Contact:

Re: How to handle high number of files?

Post by philipbehrens » Feb 21, 2019 1:58 pm

I'm using the Built-in SQL Express.

v.eremin
Product Manager
Posts: 16232
Liked: 1325 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: How to handle high number of files?

Post by v.eremin » Feb 21, 2019 2:37 pm

For such number of individual files it simply might not be enough, so consider switching to paid edition of SQL server. Thanks!

philipbehrens
Service Provider
Posts: 6
Liked: never
Joined: Jan 11, 2018 5:42 pm
Full Name: Philip Behrens
Contact:

Re: How to handle high number of files?

Post by philipbehrens » Feb 21, 2019 2:49 pm

But the veeam Task runs out of memory, not the sql server.

Dima P.
Product Manager
Posts: 9911
Liked: 790 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: How to handle high number of files?

Post by Dima P. » Feb 21, 2019 4:24 pm

Hello Philip,

Unfortunately bundled SQL Express has a limit of memory consumption per instance, so it's impossible to fit all the files you are processing into SQL database. You have several options here:

1. Split this file to tape job into several smaller file to tape jobs and schedule them to run one after anther (basically limit the number of files per job and make sure that jobs do not overlap)
2. Use SQL Standard as production database for your backup server. This option is preferable as it will let you to run file to tape jobs with huge amount of files and will increase the overall performance of file to tape jobs.

You can open the support case to investigate the issue, but I am completely sure that SQL Express limitations would be the root cause.

philipbehrens
Service Provider
Posts: 6
Liked: never
Joined: Jan 11, 2018 5:42 pm
Full Name: Philip Behrens
Contact:

Re: How to handle high number of files?

Post by philipbehrens » Feb 21, 2019 7:46 pm

Ok thank you all. I'll try it out with SQL Standard.

Dima P.
Product Manager
Posts: 9911
Liked: 790 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: How to handle high number of files?

Post by Dima P. » Feb 22, 2019 5:13 pm

Philip,

Please let us know how it goes once you upgrade your setup. Thanks!

Post Reply

Who is online

Users browsing this forum: No registered users and 6 guests