Update3 and 3par SAN snapshots being named incorrectly

[SyNtAxx]

Has anybody using the 3par SAN snapshot integration for backups under 9.5 Update3 notice the name of the snapshots are incomplete and may be causing warning/errors like: "Failed to create snapshot for LUN <lun name: There are more than 10 virtual copies for snapshot <lun name> already."

This is new behavior since my deployment to Update3 the previous day. When this happens i s seem to get errors " Failed to open VDDK <server name> read only mode -[true] errors and the vms fail.

I think the error is coming from the misidentified snapshot name(s) and some sort of counter in place to monitor the number of snapshots.

Example snap shot name vs actual name: WIN_ESX_Standard_DS19 --> snapshot name: VeeamAUX_WIN_ESX_Standard_DS1_8

Thanks,

Nick

[foggy]

Hi Nick, this doesn't look expected. Let me check with R&D. Do you have a case open, btw?

[foggy]

This appears to be caused by the max snapshot name length on 3PAR (32 characters). We name the snapshot by adding the prefix and the postfix: VeeamAUX_<vol_name>_Postfix, which results in the fact that the last digit in your example (9) is lost, since we have to make the name shorter. We will discuss if there are any options to address this.

[SyNtAxx]

Case# 02469432

This is causing me *alot* of problems, I hope R&D can get me a fix. I like the old veeam snapshot names, never had issue identifying them.

-Nick

[omegagx]

Having the same issue here - let me know when you're able to get a fix.

[SyNtAxx]

Veeam support has enabled extended logging as it pertains to 3par Snapshot communication via reg key. I should have log bundles to upload to QA/R&D tomorrow.

-Nick

[SyNtAxx]

Two new extended logging bundles have been uploaded.

-Nick

[foggy]

Current workaround is increasing the max number of virtual copies using the Hp3PARTempVirtualCopyMaxNumber registry value (current default is 10).

[SyNtAxx]

can you give the exact location of this key or where it should be created?

Nick

[foggy]

It should be created in the standard Veeam registry hive on the backup server (HKLM\SOFTWARE\Veeam\Veeam Backup and Replication).

[SyNtAxx]

ok, I'll try this. Is there an actual ceiling on the number of snapshots or upper bounds?

[SyNtAxx]

Current workaround is increasing the max number of virtual copies using the Hp3PARTempVirtualCopyMaxNumber registry value (current default is 10).

What type of key is this also ?

[SyNtAxx]

NM, I created a 32bit DWORD and seet to 20 max snaphots. Veeam accepted the new value as indicated by a new updated warning in my logs. I just updated to 40 snapshots and Hopefully that is enough to get me through until a permanent fix is created.

-Nick

[foggy]

Right, it is DWORD. We will raise the default in the next update.

[SyNtAxx]

I've upped the maximum snapshots to be 40 and the warnings/errors still persist.

-Nick

[foggy]

In other similar case I've heard about it was upped to 100 and this helped. So try to increase it more.

[SyNtAxx]

I upped the max snaphots key to 100 and I still get the warning/errors. Whats next, 200 ?

-nick

[Aumatics]

I'm about to update to update 3, but saw this post. Does this problem also occur on Hyper-V or only on VMware ?

[foggy]

I upped the max snaphots key to 100 and I still get the warning/errors. Whats next, 200 ?

Depends on how many snapshots do you have... But I'd check with support first.

[foggy]

I'm about to update to update 3, but saw this post. Does this problem also occur on Hyper-V or only on VMware ?

Hyper-V integrates with storage snapshots natively, so this functionality is for VMware only.

[SyNtAxx]

I upped the max snaphots key to 100 and I still get the warning/errors. Whats next, 200 ?

-nick

I still get errors and warnings with setting @ 200 snapshots. There is nowhere close to that many shapshots even on the SAN.

I respectfully ask for the naming convention be rolled back to before Update 3 or provide me a custom .dll or registry key to choose my snapshot name prefix. I've had to babysit my backups all day long since applying update 3. Unacceptable.

Thank you,

NIck

[foggy]

Hi Nick, I understand your frustration, further investigation is required. Naming convention was changed with the purpose of distinguishing Veeam auxiliary entities (snapshots, iSCSI hosts) created by the product and those created by customers with the testing or other purposes. The latter were commonly named Veeam_..., which could result in unintended deletion by Veeam B&R.

[cdeal]

Are there any updates on this issue? I assume that it's recommended not to install update 3 if you use 3PAR snapshot integration?

[omegagx]

We use 3PAR as well and are waiting on this issue being fixed before we upgrade to Update 3. Any news on this? Is there going to be a knowledge base article describing the issue in detail?

[foggy]

Craig, Michael, please contact technical support. They will provide the hotfix after confirming the issue in your environment.

[SyNtAxx]

All,

I belive my issue has been resolved after the application of the custom hotfix and the registry key for Max snapshots. In my case i needed to set max snapshots to 200 in order to not run into issues. The hotfix resolved the mutex timeout issues on creating/removing the snapshots. I wanted to wait on a few days of solid testing before declaring this resolved. I believe in my case the problems have been resolved.

NIck