VEEAM Snapshot Hunter does not delete VEEAM snapshots

[Kraken]

I opened a case with Veeam support (no. 01099446) ad I got the response:

Concerning snapshot not deleted by Veeam: once the communication with host is lost when Veeam tries to send a command for deleting snapshot, it would not receive any information about this snapshot on next job run, and create and delete new temporary snapshot.

Normally, snapshot indeed should be deleted with the next job run, but in some cases it does not happens due to different limitations or issues.

Unfortunately, the only way for dealing such snapshots is to delete them manually.

So yes, it might happen, when the communication is terminated unexpectedly in some moments. Not sure about the mechanism of deleting temporary snapshots by Veeam, but from what I see, these snapshots are monitored by Veeam and information is stored in internal database - when job runs next time and Veeam knows (based upon its own database), that there is temporary snapshot, it is indeed deleted.

But there is no "search for snapshot with the 'Veeam Temporary snapshot' name on the machine that is going to be backed up and if there is one, delete this snapshot and then proceed normally" - which is the mechanism, that is used by the "script approach" to manually delete temporary snapshots.

In general, this can be monitored, but since we backup a lot of Windows machines that runs a lot of processes, then even with VSS enables, sometimes the Application aware processing is not possible, so the backup job fails in the first run quite often, but succeed in second. I have mail notifications, when job fails after third attempt, so now I think it will be necessary to get a note after every failure and check the temporary snapshots.

[Gostev]

Looks like your issue is caused by the same flaw in the logic around handling connection loss that has already been addressed in v9.

[ITP-Stan]

I think it was wrong to call this feature Snapshot Hunter in V8 when clearly it does not hunt for snapshots.
Another customer, similar problem.
https://onedrive.live.com/redir?resid=F ... hoto%2cjpg

[foggy]

I think it was wrong to call this feature Snapshot Hunter in V8 when clearly it does not hunt for snapshots.

As it was already stated a couple of times earlier in this thread, visible VEEAM BACKUP TEMPORARY SNAPSHOTS are outside of the scope of Snapshot Hunter functionality, and are rather managed by the backup job itself.

[ITP-Stan]

I think it's strange that you would design a feature to hunt for hidden/stuck snapshots, but do not have some kind of safety logic that will remove snapshots visible but left behind by the backup job. Or that the backup job would not even fail but keeping putting on new snapshots which again it doesn't delete.

I know the repsonse to this already, must be specific to your environment, since none of our customers have reported this. But here I am, reporting it again voor another customer. It's not because client or resellers solve the problem themselves, that it doesn't exist.

[Vitaliy S.]

but do not have some kind of safety logic that will remove snapshots visible but left behind by the backup job.

Actually we do have such logic, and it was also explained following the link in the previous post. Our jobs will remove temporary snapshots if they were left behind by previous job runs. That's why it is recommended to investigate this possible environment specific issue with our support team.

[ITP-Stan]

As you can read in this topic, the last time the support team checked it.
The ticket was open for a couple of weeks, with many logs exported and send every time they were requested. With escalation engineers, asking for logs yet again. After 2 weeks the reply was that developers say that after some failed attempts VEEAM assumes that the snapshots are deleted and does not recheck this at the start of the next job. They might fix it in a future version. So unless I have been given false information there isn't really a safety mechanism for these snapshots.

I will not open another support case, as the last one required many hours of assistance, we can't charge the customer for, only to get the answer that it can't be fixed and might be in some future version.

Reference:

The escalation engineer has escalated the case to the next tier How many levels can I escalate ...

The case will be closing with a response that in future versions they will fix this: But no answer what future version.

[pirx]

Well... temporary VM snapshots should be removed by a subsequent job run.

I have just checked your case, and our support engineer stated that snapshot hunter will not try to remove this snapshot, however the next job run should do it. Currently it is not clear why this snapshot has not been removed, as logs do not contain information about that date and other job runs don't have it either. Let's check if new logs package has this data.

While we are waiting, I would recommend to remove it manually, so that it doesn't grow much and doesn't affect your production, then install free version of Veeam ONE to monitor VM snapshots in your infrastructure.

If you have the same situation in future, open a support case and post your case ID here.

Case 02541546

We had NTP and AD issued two days ago, jobs failed. Later after issues were solved, retries of the jobs were successful. Next day more than 200 Veeam temp. snapshots were reported, all from the time of the issues with AD. Next backup cycle did not clean this up.

I'm not very happy with support just pointing out to delete the snaps on my own. There seems to be not much interest in finding out, why Hunter does not detect/remove them. I'd also appreciate if Veeam would provide a script to clean up this mess, or even better a tool other than Snapshot Hunter to do this in more automated way. I've now to decide if I go an delete everything manually or try to get a scrip done. But snaps are growing so there is not unlimited time.

In your case it seems the Snapshot Hunter is unable to remove the snpashot

https://helpcenter.veeam.com/docs/backu ... ml?ver=120

Please consult internally if it is alright to go ahead and remote the snapshots from vSphere:

https://kb.vmware.com/s/article/1023657

If you are not sure of the process I strongly recommend reaching out to VMWare support to assist you with this, just to be on the safe side.

[AlexHeylin]

We've also had issues with VBR leaving its temp snapshots behind. I can't be sure if the next run removed them or not, but it would be nice if when a backup or replication which has created a snapshot job fails (e.g. a server gets restarted for patching etc) that it would come back and remove the snapshot without waiting for the next run of the job which is likely to be at least a day away, possibly more. I'll try to track if the next run does / doesn't remove them in our case.