Enterprise Manager indexing/backup issue

[ChrisGundry]

Hi All,

Wondering if anyone on the forum team can assist with this as I am not getting anywhere with support... Case #03175124.

Basically we have our service desk with access to EM portal to restore files for users. The service desk don't have the remote console installed.

The service desk recently reported that they were trying to restore a backup from a couple of weeks ago but the restore button was greyed out. Seemed like a permissions issue. I checked the ability to restore from that point myself and it was also greyed out for me, I have full access.

Then I realised that this was a point on a Wednesday, 2 weeks ago. We only retain daily backups of this system for 7 days, so this point didn't exist in the repo... I checked the repo and sure enough the file isn't there.

The EM console shows a backup available for every day over the last month. This isn’t the case, we only have backups daily for the last 7 days, then weekly and monthly etc after that. We have daily storage snapshots available, but they are not available within EM and they are taken at a different time to the time shown on the ‘backup’ that is listed within EM.

When I click on the backup info bit for the backup in question in the EM portal it indicates this is a tape backup file. We don't have tape backups...

By chance I checked the self-service portal and the restore points listed in there are correct, only 7 days of daily, then weekly, monthly etc.

The EM index setting is set to 1 months and from what I have worked out this seems to be the cause of the restore points showing in EM.

So I have a couple of questions:
1. Why does this EM index setting exist? I don’t see why you would want to see the index/backup content of a backup that no longer exists on disk…
2. Why is there no option to say “only retain index data for backups that exist on disk” or “only show me backups that actually exist”?
3. Why does self-service show the correct/available backups but EM doesn’t?
4. If we set the EM index setting to 12 months, it seems to keep the daily backup index files for 12 months, even though the backups are deleted by retention after 7 days (as per issues above). Therefore we have had to set it to the minimum of 1 month to reduce the disk space usage. This means we now don’t have indexes for the weekly and monthly backups that are over 1 month old.

To me this EM index feature doesn’t have a purpose and is causing us a major headache. So I feel like I must be missing a fundamental reason for this function to exist…? Any feedback appreciated!

Thanks

[foggy]

1. This setting exists to allow restores from backups that are out of retention and moved to a secondary/archive storage/tape/etc. (that's why you see the tape icon there).
2. This might be a good addition, so let's consider this as a feature request.
3. Self-service portal implies a different scenario, where there's no administrator to put in the required tape, for example.

[ChrisGundry]

Hi Foggy,

Thanks for the reply, but it doesn't really answer my questions/issue.
1. Why does this EM index setting exist? I don’t see why you would want to see the index/backup content of a backup that no longer exists on disk… - You say this is to show archived/tape backups etc. But this backup mentioned doesn't exist, anywhere, it is deleted by retention. So you can't put a tape or anything in to restore it... So why is it showing it?

2. Why is there no option to say “only retain index data for backups that exist on disk” or “only show me backups that actually exist”?
Yeah, I don't know why its not part of the current setup to be honest...

3. Why does self-service show the correct/available backups but EM doesn’t?
Right OK, I guess that makes sense.

4. If we set the EM index setting to 12 months, it seems to keep the daily backup index files for 12 months, even though the backups are deleted by retention after 7 days (as per issues above). Therefore we have had to set it to the minimum of 1 month to reduce the disk space usage. This means we now don’t have indexes for the weekly and monthly backups that are over 1 month old.

4 is where it really falls down to me... Keeping daily backup index files once the backups don't exist means that the size of the index data gets huge. I don't understand how anyone can be keeping indexes for longer than a month, unless they have a really small estate? We have a fairly decent sized file server, but not massive by any means. We only have indexing enabled for a month and the indexes are taking up 15GB of space. But that is only for the daily backups, we have lost our indexes for the monthly GFS backups because we have had to set the index retention to only 1 month, otherwise as the indexes were growing month by month we were losing 15GB per month of space!

[foggy]

1. Veeam B&R might not know where this backup finally went (it could be offloaded on some external media manually), but still keeps indices according to the configured retention - for the case where these backups are brought back from archive and imported into Veeam B&R.

4. Haven't you thought of enabling NTFS compression on the volume where indices are stored?

[ChrisGundry]

1. I get what you are saying, but in this case Veeam itself deleted the backup for retention. So it knows it was deleted and not archived to tape/disk etc. manually...

4. I did consider that, but didn't do it for a couple of reasons:
1. If Veeam wasn't doing this odd process in the first place we wouldn't need to. Why keep indexes for backups 365+ daily backups that don't exist, just so we can retain indexes for the few backups we actually keep on disk...?
2. Most software vendors don't support compression/compressed volumes/folders and blame that whenever there are issues. For example, I suspect it would slow down loading the Veeam index within EM?
3. I checked and the data within 90% of the index folders is already in a .zip folder (GuestIndexData.zip) so compresison doesn't help. Folders are around 300Mb each/per day.

A couple of the folders in the index area have files within internal\src folder called, for example, volume.c_.ifd, volume.d_.ifd, volume.g_.ifd, which is over 1.5GB in total. The other folders all have only GuestIndexData.zip which contains the zipped version of these files. So for some reason it doesn't seem to have zipped up a couple of folders on a couple of occasions.

We want to be able to use EM and restore from all primary and secondary GFS backups as intended (I guess?), meaning we need a year or more of indexes available as our backups go back several years. At 15GB per month that is 180GB ish for just 12 months!! That is crazy! But most of those indexes are actually for backups that don't exist anymore...

What I am saying is that 2/3rd of the daily indexes it is retaining are of no use because the backups don't exist on disk anymore and won't ever again because Veeam has deleted them by retention policy. So our 15GB month of archive should only really be about <5GB.

If you removed the backups/days that don't have a backup on disk anymore you would reduce the number of indexes needed by 2/3rd in our case. So the indexes would go down from 15GB in month 1 to <5GB in month 1. Month 2+ would have only 4 weekly or 1 monthly points/indexes so would be <1GB most months, so a year of indexes for us would be would be massivly reduced from 180GB down to under 20GB!

Considering that most of those indexes are for days/backups that don't exist anymore I just can't justify wasting that amount of space on a 'feature' that I don't think should exist in the first place. We can't even turn it off, so we don't waste the 15GB we are wasting at the moment, the lowest it goes is 1 month. It would be much better if we had an option so that it only retains indexes for backups that still exist on disk in primary or secondary repos.

[foggy]

So for some reason it doesn't seem to have zipped up a couple of folders on a couple of occasions.

If I'm not mistaken, it should maintain 14 days of uncompressed data and compress everything that is older.

It would be much better if we had an option so that it only retains indexes for backups that still exist on disk in primary or secondary repos.

Yes, your request is reasonable.

[ChrisGundry]

That might explain it, although I think it is only 3 of the days, not 14 which is odd...

So can we get it as a proper feature request? At the moment we can't use this functionality at all due to the size of the index files that are retained. Thanks

[foggy]

Having posted here you made a feature request, but we cannot comment on any ETA for it, since we prioritize features based on multiple factors, including the number of requests and value to the product. Thanks!

[ChrisGundry]

Hi Foggy,

Can you please advise if there has been any further discussion around this issue? I wasn't able to get anywhere with support on the issue, they said it was working as intended (which makes no sense to us) and we had to raise a feature request to do anything further.

We are running low on free disk space as a result of having to keep indexes for backups that don't exist. If we don't keep the indexes then our service desk team and users can't use the portal to restore files. If we keep the indexes then it retains indexes for backups that don't exist anymore, taking up valuable space.

Thanks

[foggy]

Nothing new in this regard. Decreasing the retention period for indices a bit more or increasing available space look like the most viable workarounds so far.

[TomQB]

Hi,

I'm also interested in this feature. The workaround is not a good solution when you plan to keep the index for longer time.
Any other workaround or something new about this feature?

[ChrisGundry]

FYI we are still struggling with this issue and know no workarounds. We are at the point where we are turning off Enterprise Manager functionality because of this issue.

[foggy]

So far this feature hasn't got high enough in the priority list.

[SBarrett847]

A bump for the priority list. Showing of the historical indexes as available, for unavailable backup restore points, is making the EM useless for our help desk.

[FXE]

Hi all !

This is a feature request about Guest files system behaviour in Veeam Backup Enterprise Manager.

3 days ago, I opened a case (05100283) about this because, in guest files recovery, it's frustrating to have to search randomly in calendar the oldest valid restore point.
For example, in our case, we have a backup retention of 40 days and and guest files system catalog retention of 3 months.
So, it's not practical on VEM calendar because recovery points oldest than backup retention period are shown. And when we want to recover the oldest recovery point for a file in particular, we have to navigate between several recovery points to find the oldest real recovery point.

Finally, my request : would it be possible to have a setting to show only valid recovery points or to align guest files system retention to backup retention ?

Thank you !

[HannesK]

Hello,
I merged your question to the existing feature request... it's possible yes, but it depends on development resources.

Best regards,
Hannes

[ChrisGundry]

It's ironic this has come back up in my notifications. We had another requirement last week which would have been fulfilled by dev ops team having access to EM, but because of this ridiculous indexing issue they can't have access, meaning we have to do all their restores manually for them. Sad to see this is still an issue several years later!