Feature Request for large number of files

[jamcool]

I was a TSM (Tivoli Storage Manager) admin for about 15 years before we retired it a few years ago and replaced it with Veeam's Windows agent. One feature that TSM client on Windows had was called Journaling. It would track files that changed so that when you did a file level backup, it just had to backup the files it had logged and not inspect every single file on the server. This was great for systems that had millions of files. It had a limitation such that if you stopped TSM journaling service (or rebooted), it had to do one regular backup (inspect all files) but after that it just checked the journal's log file. Backups that would take days to run inspecting millions (or 10s of million of files) would run in minutes. Would be nice to have a feature like this in Veeam's Window agent. Most of my large number of files servers have been replaced by NAS but I still have a few that have those 1+ million files and take half a day to run and backup maybe a thousand files of less than 50 GB in the end.

[HannesK]

Hello,
that feature was great long time ago, yes.

Today we have block based backup with changed block tracking (CBT).

I recommend using "entire system" or "volume level backup". That's what the agent is built for. https://helpcenter.veeam.com/docs/backu ... ml?ver=110

If you really want file-based backup, we have NAS backup for that. Different mechanism (reboots are no problem), but also very fast. https://helpcenter.veeam.com/docs/backu ... ml?ver=110

Best regards,
Hannes

[Gostev]

For image-level backups, also be sure you have the Veeam CBT driver installed on the protected machine.

[jamcool]

The CBT option did not work. We are doing File level backup as they only want the folders with data in it backed up. Perhaps that is why it is not working? It still sits there for hours before that small of backup happens.

[HannesK]

Hello,
CBT (change block tracking) can only work with block based backup. If you chose file based backup, then it's "no change tracking" except you use NAS backup (see my post above).

Usually it's not worth thinking about the overhead of unnecessary backup. File based backup usually takes more space than the "savings" (and you cannot forget anything). Example: 1KB of a 1GB file changed... that is 1GB new backup with file-based backup, but only 1 MB (or whatever block size is configured) for block based backup.

It sounds like your server is too large for VSS exclusions. If you can exclude the folders, then volume based backup can also work https://www.veeam.com/kb3236 (that is block based). But usually that only scales up to 500k - 1M files depending on the Windows version and unknown conditions (it's unpredictable)

Best regards,
Hannes