Discussions related to exporting backups to tape and backing up directly to tape.
Post Reply
philipbehrens
Service Provider
Posts: 6
Liked: never
Joined: Jan 11, 2018 5:42 pm
Full Name: Philip Behrens
Contact:

How to handle high number of files?

Post by philipbehrens »

Hi,

currently I am facing problems with a file to tape backup job.
The Action "Building protected objects list" took nearly infinity until the backup server runs out of RAM and crashes.
Possibly this is because of the high number of files (12 TB - ~12.000.000 Files).
Is there a way to handle this amount?
Gostev
Chief Product Officer
Posts: 31428
Liked: 6633 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: How to handle high number of files?

Post by Gostev »

High, what version are you using?
philipbehrens
Service Provider
Posts: 6
Liked: never
Joined: Jan 11, 2018 5:42 pm
Full Name: Philip Behrens
Contact:

Re: How to handle high number of files?

Post by philipbehrens »

9.5.4.2615
Standard Edition
veremin
Product Manager
Posts: 20261
Liked: 2249 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: How to handle high number of files?

Post by veremin »

I'm wondering what type of SQL server you're using. Built-in SQL Express or fully blown SQL server. Thanks!
philipbehrens
Service Provider
Posts: 6
Liked: never
Joined: Jan 11, 2018 5:42 pm
Full Name: Philip Behrens
Contact:

Re: How to handle high number of files?

Post by philipbehrens »

I'm using the Built-in SQL Express.
veremin
Product Manager
Posts: 20261
Liked: 2249 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: How to handle high number of files?

Post by veremin »

For such number of individual files it simply might not be enough, so consider switching to paid edition of SQL server. Thanks!
philipbehrens
Service Provider
Posts: 6
Liked: never
Joined: Jan 11, 2018 5:42 pm
Full Name: Philip Behrens
Contact:

Re: How to handle high number of files?

Post by philipbehrens »

But the veeam Task runs out of memory, not the sql server.
Dima P.
Product Manager
Posts: 14388
Liked: 1566 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: How to handle high number of files?

Post by Dima P. »

Hello Philip,

Unfortunately bundled SQL Express has a limit of memory consumption per instance, so it's impossible to fit all the files you are processing into SQL database. You have several options here:

1. Split this file to tape job into several smaller file to tape jobs and schedule them to run one after anther (basically limit the number of files per job and make sure that jobs do not overlap)
2. Use SQL Standard as production database for your backup server. This option is preferable as it will let you to run file to tape jobs with huge amount of files and will increase the overall performance of file to tape jobs.

You can open the support case to investigate the issue, but I am completely sure that SQL Express limitations would be the root cause.
philipbehrens
Service Provider
Posts: 6
Liked: never
Joined: Jan 11, 2018 5:42 pm
Full Name: Philip Behrens
Contact:

Re: How to handle high number of files?

Post by philipbehrens »

Ok thank you all. I'll try it out with SQL Standard.
Dima P.
Product Manager
Posts: 14388
Liked: 1566 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: How to handle high number of files?

Post by Dima P. »

Philip,

Please let us know how it goes once you upgrade your setup. Thanks!
Post Reply

Who is online

Users browsing this forum: devlopur and 8 guests