Standalone backup agent for Microsoft Windows servers and workstations (formerly Veeam Endpoint Backup FREE)
Post Reply
jamcool
Enthusiast
Posts: 67
Liked: 11 times
Joined: Feb 02, 2018 7:56 pm
Full Name: Jason Mount
Contact:

Feature Request for large number of files

Post by jamcool »

I was a TSM (Tivoli Storage Manager) admin for about 15 years before we retired it a few years ago and replaced it with Veeam's Windows agent. One feature that TSM client on Windows had was called Journaling. It would track files that changed so that when you did a file level backup, it just had to backup the files it had logged and not inspect every single file on the server. This was great for systems that had millions of files. It had a limitation such that if you stopped TSM journaling service (or rebooted), it had to do one regular backup (inspect all files) but after that it just checked the journal's log file. Backups that would take days to run inspecting millions (or 10s of million of files) would run in minutes. Would be nice to have a feature like this in Veeam's Window agent. Most of my large number of files servers have been replaced by NAS but I still have a few that have those 1+ million files and take half a day to run and backup maybe a thousand files of less than 50 GB in the end.
HannesK
Product Manager
Posts: 14287
Liked: 2877 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: Feature Request for large number of files

Post by HannesK » 2 people like this post

Hello,
that feature was great long time ago, yes.

Today we have block based backup with changed block tracking (CBT).

I recommend using "entire system" or "volume level backup". That's what the agent is built for. https://helpcenter.veeam.com/docs/backu ... ml?ver=110

If you really want file-based backup, we have NAS backup for that. Different mechanism (reboots are no problem), but also very fast. https://helpcenter.veeam.com/docs/backu ... ml?ver=110

Best regards,
Hannes
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Feature Request for large number of files

Post by Gostev » 1 person likes this post

For image-level backups, also be sure you have the Veeam CBT driver installed on the protected machine.
jamcool
Enthusiast
Posts: 67
Liked: 11 times
Joined: Feb 02, 2018 7:56 pm
Full Name: Jason Mount
Contact:

Re: Feature Request for large number of files

Post by jamcool »

The CBT option did not work. We are doing File level backup as they only want the folders with data in it backed up. Perhaps that is why it is not working? It still sits there for hours before that small of backup happens.
HannesK
Product Manager
Posts: 14287
Liked: 2877 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: Feature Request for large number of files

Post by HannesK »

Hello,
CBT (change block tracking) can only work with block based backup. If you chose file based backup, then it's "no change tracking" except you use NAS backup (see my post above).

Usually it's not worth thinking about the overhead of unnecessary backup. File based backup usually takes more space than the "savings" (and you cannot forget anything). Example: 1KB of a 1GB file changed... that is 1GB new backup with file-based backup, but only 1 MB (or whatever block size is configured) for block based backup.

It sounds like your server is too large for VSS exclusions. If you can exclude the folders, then volume based backup can also work https://www.veeam.com/kb3236 (that is block based). But usually that only scales up to 500k - 1M files depending on the Windows version and unknown conditions (it's unpredictable)

Best regards,
Hannes
Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 17 guests