How to handle high number of files?

[philipbehrens]

Hi,

currently I am facing problems with a file to tape backup job.
The Action "Building protected objects list" took nearly infinity until the backup server runs out of RAM and crashes.
Possibly this is because of the high number of files (12 TB - ~12.000.000 Files).
Is there a way to handle this amount?

[Gostev]

High, what version are you using?

[philipbehrens]

9.5.4.2615
Standard Edition

[veremin]

I'm wondering what type of SQL server you're using. Built-in SQL Express or fully blown SQL server. Thanks!

[philipbehrens]

I'm using the Built-in SQL Express.

[veremin]

For such number of individual files it simply might not be enough, so consider switching to paid edition of SQL server. Thanks!

[philipbehrens]

But the veeam Task runs out of memory, not the sql server.

[Dima P.]

Hello Philip,

Unfortunately bundled SQL Express has a limit of memory consumption per instance, so it's impossible to fit all the files you are processing into SQL database. You have several options here:

1. Split this file to tape job into several smaller file to tape jobs and schedule them to run one after anther (basically limit the number of files per job and make sure that jobs do not overlap)
2. Use SQL Standard as production database for your backup server. This option is preferable as it will let you to run file to tape jobs with huge amount of files and will increase the overall performance of file to tape jobs.

You can open the support case to investigate the issue, but I am completely sure that SQL Express limitations would be the root cause.

[philipbehrens]

Ok thank you all. I'll try it out with SQL Standard.

[Dima P.]

Philip,

Please let us know how it goes once you upgrade your setup. Thanks!