Pure Storage snapshot job completes with a warning because of Pure's safemode retention period

[kjstech]

Posting in response to case #07703308 with Veeam Backup & Replication Support.

[Moderator: Replaced Support ID with Case ID]

Veeam B&R with the Pure Storage Plugin does some Storage snapshot based backup jobs. Basically Veeam orchestrates with Pure's API to have the array do all the heavy lifting, vs traditional backups using VMWares storage API and longer, more CPU intensive backups.

Once the storage snapshot is done, Veeam tries to delete the snapshot it created on the pure. But with Pure safemode, the snapshot goes into a "deleted" area where it will not actually be deleted for 7 days. This is data protection so if a rouge admin or bad actor got into the system, they could not encrypt your data and then delete all the good backups. Pure Support is the only other "key turner" with other verified people from your org to make any changes to the safemode protection.

For this reason all pure storage snapshot jobs in my environment complete with a warning. I would love a simple checkbox in the Pure storage plug-in or Veeam to "ignore errors related to storage snapshot deletion" for each job.

All the VM's in the volume are backed up successfully in only a few minutes time, but the entire job is marked warning due to a message similar to this:
"Deleting storage snapshot purearray-hq:Tier5-DevTest.VEEAM-ProdSnap-PureSnapshot-Tier5-1H1ERRvmJuEUoGoAeqd-owzOOYqoz on volume purearray-hq:Tier5-DevTest according to the Pure Storage FlashArray Replication snapshot retention policy. "

In this example, if I go to the Pure Flasharray C20, I certainly see the snapshot in the deleted area with an eradication timer counting down 7 days, so many hours and minutes, etc... It will go away on its own after that time is up, so there really isnt a reason for Veeam to mark the job with a warning.

If Safemode is NOT enabled, the job completes in green with a "success". Veeam can delete the temporary snapshot immediately. However this is risky as a rouge admin or bad actor could do the same.

Please consider in a future version of veeam, a small checkbox to ignore the snapshot deletion error.

[david.domask]

Hi kjstech,

Thank you for sharing the case number and sorry to hear about the challenges.

What version of the plugin are you on? If I remember right, version v2.1.388 of the plugin from https://www.veeam.com/kb4129 has logic to accommodate for this, and previous version of the plugin was supposed to address quite a few of these issues with safe mode and the clones made for backup from storage snapshot.

[markdpolin]

Hi kjstech,

Thank you for your post. I am working with Pure Storage USAPI plugin product management to see if this warning can be addressed in a future release of the Pure Storage USAPI plugin.

[kjstech]

Hi, We are using Pure Storage plugin 2.1.388.0.

Everything appears to work fine, the plugin does move the snapshot to the "deleted" area, but then the eradication timer begins its countdown (for us 7 days). The Pure storage array will clean itself up, but I think because of this its a simple warning on the backup job.

While the backups do perform, whenever we see warnings (and obviously failures), we look at the job to ensure it completed. If we see the success message in our outlook Veeam folder, then at a glance we know 100% everything ran well.

[foggy]

The updated plug-in addressing this issue was released last week - see version 2.1.400. Just please note that, unlike version 2.1.388, it requires Veeam Backup & Replication 12.3 or later.