-
- Veteran
- Posts: 489
- Liked: 93 times
- Joined: Sep 17, 2017 3:20 am
- Full Name: Franc
- Contact:
NetApp tamperproof snapshots cause veeam_aux volume not being deleted
Case #07108016
On our NetApp system running Ontap 9.13.1 we’ve enabled the new tamperproof snapshot copies feature. We’ve set a snapshot schedule to automatically create snapshots every hour. As a test we’ve set the Snaplock retention to 1 day.
For backups using Veeam we use storage integration.
The problem we are facing now is that Veeam sometimes is unable to delete its storage snapshot because it’s locked by the Snaplock retention. This happens when Veeam creates its snapshot around the time the automatic snapshot schedule on the NetApp runs and creates a tamperproof snapshot. By default a volume clone inherits its parents snapshot settings. This results in automatic tamperproof snapshots being taken of the temporary clone also.
In NetApp system manager we now have multiple veeam_aux volumes which can’t be deleted yet since the Snaplock retention has not expired.
The affected Veeam jobs return the following warning:
Failed to delete snapshot VeeamAUX<volme name> of volume VeeamAUX<volume name>. Details: Failed to delete volume. The volume has one or more unexpired locked Snapshot copies.
The solution would be that Veeam removes the snapshot schedules from the cloned volume so that no automatic snapshots will be created and Veeam can delete the volume after the backup completes.
The situation currently is that you end up with many veeam_aux volumes on the storage system which can only be deleted when the retention expires. Depending on the Snaplock configuration this can take weeks or months. Also these volumes need to be manually deleted. Not a pleasant situation to say the least
On our NetApp system running Ontap 9.13.1 we’ve enabled the new tamperproof snapshot copies feature. We’ve set a snapshot schedule to automatically create snapshots every hour. As a test we’ve set the Snaplock retention to 1 day.
For backups using Veeam we use storage integration.
The problem we are facing now is that Veeam sometimes is unable to delete its storage snapshot because it’s locked by the Snaplock retention. This happens when Veeam creates its snapshot around the time the automatic snapshot schedule on the NetApp runs and creates a tamperproof snapshot. By default a volume clone inherits its parents snapshot settings. This results in automatic tamperproof snapshots being taken of the temporary clone also.
In NetApp system manager we now have multiple veeam_aux volumes which can’t be deleted yet since the Snaplock retention has not expired.
The affected Veeam jobs return the following warning:
Failed to delete snapshot VeeamAUX<volme name> of volume VeeamAUX<volume name>. Details: Failed to delete volume. The volume has one or more unexpired locked Snapshot copies.
The solution would be that Veeam removes the snapshot schedules from the cloned volume so that no automatic snapshots will be created and Veeam can delete the volume after the backup completes.
The situation currently is that you end up with many veeam_aux volumes on the storage system which can only be deleted when the retention expires. Depending on the Snaplock configuration this can take weeks or months. Also these volumes need to be manually deleted. Not a pleasant situation to say the least
-
- Product Manager
- Posts: 8735
- Liked: 2296 times
- Joined: May 13, 2017 4:51 pm
- Full Name: Fabian K.
- Location: Switzerland
- Contact:
Re: NetApp tamperproof snapshots cause veeam_aux volume not being deleted
Hi Franc
Veeam Backup & Replication is not yet aware of NetApp tamperproof snapshots. Therefore I expect such side affects. We have official support on our roadmap, but no ETA yet.
When we introduce official support, we will make sure that such use cases are also covered.
Best,
Fabian
Veeam Backup & Replication is not yet aware of NetApp tamperproof snapshots. Therefore I expect such side affects. We have official support on our roadmap, but no ETA yet.
When we introduce official support, we will make sure that such use cases are also covered.
Best,
Fabian
Product Management Analyst @ Veeam Software
-
- Veteran
- Posts: 489
- Liked: 93 times
- Joined: Sep 17, 2017 3:20 am
- Full Name: Franc
- Contact:
Re: NetApp tamperproof snapshots cause veeam_aux volume not being deleted
ok, thanks. In the meantime I've contacted NetApp support to ask if it's possible to disable the automatic inheritance of snapshot schedules when mounting a snapshot as a temporary volume.
-
- Veteran
- Posts: 489
- Liked: 93 times
- Joined: Sep 17, 2017 3:20 am
- Full Name: Franc
- Contact:
Re: NetApp tamperproof snapshots cause veeam_aux volume not being deleted
Netapp support stated that it’s not possible to disable the automatic inheritance of the tamperproof snapshot settings and schedule. So for now, it’s not possible for us to activate this feature until Veeam is compatible with it. Bummer…
-
- Veeam Software
- Posts: 21073
- Liked: 2115 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: NetApp tamperproof snapshots cause veeam_aux volume not being deleted
Hi Franc, we've researched this a bit and it is possible to disable automatic schedule inheritance right after creating the clone on the Veeam B&R side. We will implement this in one of the upcoming updates as a workaround/fix for the behavior you're seeing. Thanks for the heads up!
-
- Veteran
- Posts: 489
- Liked: 93 times
- Joined: Sep 17, 2017 3:20 am
- Full Name: Franc
- Contact:
Re: NetApp tamperproof snapshots cause veeam_aux volume not being deleted
Ok, great!
I think you may also need to clear the 'enable snapshot locking' setting on the cloned volume or else the volume still can't be deleted even when there are no locked snapshots present.
I think you may also need to clear the 'enable snapshot locking' setting on the cloned volume or else the volume still can't be deleted even when there are no locked snapshots present.
-
- Veteran
- Posts: 489
- Liked: 93 times
- Joined: Sep 17, 2017 3:20 am
- Full Name: Franc
- Contact:
Re: NetApp tamperproof snapshots cause veeam_aux volume not being deleted
thanks, we aren't using snapmirror, but these were exactly the steps I had to take earlier to be able to delete the _aux volumes.
-
- Veeam Software
- Posts: 21073
- Liked: 2115 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: NetApp tamperproof snapshots cause veeam_aux volume not being deleted
This may be a different issue, e.g. this one: https://kb.netapp.com/onprem/ontap/dp/S ... hot_copies
Without a schedule being inherited, the clone volume shouldn't get locked and can be deleted even with the 'Enable snapshot locking' setting enabled - at least this is what we're seeing on 9.14.
Without a schedule being inherited, the clone volume shouldn't get locked and can be deleted even with the 'Enable snapshot locking' setting enabled - at least this is what we're seeing on 9.14.
-
- Veteran
- Posts: 489
- Liked: 93 times
- Joined: Sep 17, 2017 3:20 am
- Full Name: Franc
- Contact:
Re: NetApp tamperproof snapshots cause veeam_aux volume not being deleted
We are on 13.1P6 and are planning to install P7. 9.14.1 is quite new, so we wait a couple of patches before updating to that one.
Who is online
Users browsing this forum: No registered users and 112 guests