Why Veeam B&R read files before deleting them

[sirjeje]

Hello,

We used a specific device for our repository : files are on disk for 15days, and then they are only on tapes (but they are still visible for "users" as if they were on disk).
The issue is for the cleaning of files that match our expiration policy.
When it happens (at the end of a job), Veeam, before deleting the obsolete files, tries to read the file. But in our configuration, the file to read is on tape => the job is very long to finish.
We opened a case for this (07071195), but the response is : veeam need to read the file before deleting it.
I think it's very strange that veeam need to read the file enterily, just for deleting a file that is expired. Maybe just a header of the file (or even better, delete without reading)

Any thought on this?

[Mildur]

Hi Sirjeje

Unfortunately the case was not in English.
So I cannot understand what was discussed between you and the support engineer.

But I understood from some of the mails, that you are using a storage device which does move backup files away from the disk storage to tapes outside of Veeam? The backup files must be available for the backup server. We do not support <lifecycle> policies when those are not managed by the backup server. This can always lead to issues. You should led the backup server use a backup to tape to send the write to tape.

May I ask, what backup method do you use for the backups? Forever Forward or Forward incremental?

Best,
Fabian

[sirjeje]

Yes sorry about the french.
The files are not "moved" outside of Veeam, there are still on the Veeam repository, and accessible directly by Veeam.
The point is somes files will be available directly during a laps of time, and the rest will be available after a read on a tape that puts the files on the hard drive.

We do one full each week, and then increment the others days.

I don't see the point of reading entirely the files before deleting them. To see if the files exists, I understand. I also understand for an header on the files before deleting.

Also for information, we use this kind of storage since 6 years, not a single issue since the opening of this ticket.

[Andreas Neufert]

Hello Sirjeje,

thanks for reaching out.
I can explain a bit further. Veeam manages the backup files and we are always checking for the integrity of our backup chains. This is as well true when we delete files, we want to be sure that these files represent the specific backup files and not just "any" file that has the same name. For this reason rescan, deletes and other operations work actively with the backup files metadata information within the backups files. If something is broken or not correct, we mark the backup chain to "repair" state, to not harm the data until this is resolved. This can happen if we timeout at the delete processing or we can not delete specific files.

Our documentation points out that we do not support gateway systems that move things to other medias like tape:
https://helpcenter.veeam.com/docs/backu ... -and-tapes

You have multiple options:
1) Keep 100% of the data on the filer area (your system just mirror everything to tape)
2) Use Veeam's Tape Job capabilities to maintain backups on the disk system and use Veeam to maintain the backup chain on Tape.
3) Use Object Storages that support Glacier like offloading to tape. Select a system that leverage Veeam's Archive Tier SOSAPI integration. For example POINT, Quantum and SpectraLogic to name some.

[sirjeje]

Hello,

Thanks for your response.
I understand your point (as you explain well, even if in our workflows this is not relevant) . Could you please indicate where are the meta data on those backups files? At the begining? At the end?
As I think we could manage it well by just having this information.

Thanks.

[Andreas Neufert]

I am not sure on what and how we read in your specific situation.
In general are the first few MB the header of the file that then reference to metadata banks that are spread across the whole file. Maybe first 64MB might be enough for your case, but this is just a guess.

[sirjeje]

Thanks for the information!