Comprehensive data protection for all workloads
Post Reply
SyNtAxx
Expert
Posts: 149
Liked: 15 times
Joined: Jan 02, 2015 7:12 pm
Contact:

Update3 and 3par SAN snapshots being named incorrectly

Post by SyNtAxx »

Has anybody using the 3par SAN snapshot integration for backups under 9.5 Update3 notice the name of the snapshots are incomplete and may be causing warning/errors like: "Failed to create snapshot for LUN <lun name: There are more than 10 virtual copies for snapshot <lun name> already."

This is new behavior since my deployment to Update3 the previous day. When this happens i s seem to get errors " Failed to open VDDK <server name> read only mode -[true] errors and the vms fail.

I think the error is coming from the misidentified snapshot name(s) and some sort of counter in place to monitor the number of snapshots.

Example snap shot name vs actual name: WIN_ESX_Standard_DS19 --> snapshot name: VeeamAUX_WIN_ESX_Standard_DS1_8

Thanks,

Nick
foggy
Veeam Software
Posts: 21138
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by foggy »

Hi Nick, this doesn't look expected. Let me check with R&D. Do you have a case open, btw?
foggy
Veeam Software
Posts: 21138
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by foggy »

This appears to be caused by the max snapshot name length on 3PAR (32 characters). We name the snapshot by adding the prefix and the postfix: VeeamAUX_<vol_name>_Postfix, which results in the fact that the last digit in your example (9) is lost, since we have to make the name shorter. We will discuss if there are any options to address this.
SyNtAxx
Expert
Posts: 149
Liked: 15 times
Joined: Jan 02, 2015 7:12 pm
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by SyNtAxx »

Case# 02469432

This is causing me *alot* of problems, I hope R&D can get me a fix. I like the old veeam snapshot names, never had issue identifying them. :)

-Nick
omegagx
Enthusiast
Posts: 68
Liked: 4 times
Joined: May 09, 2017 6:33 pm
Full Name: Mike G
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by omegagx »

Having the same issue here - let me know when you're able to get a fix.
SyNtAxx
Expert
Posts: 149
Liked: 15 times
Joined: Jan 02, 2015 7:12 pm
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by SyNtAxx »

Veeam support has enabled extended logging as it pertains to 3par Snapshot communication via reg key. I should have log bundles to upload to QA/R&D tomorrow.

-Nick
SyNtAxx
Expert
Posts: 149
Liked: 15 times
Joined: Jan 02, 2015 7:12 pm
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by SyNtAxx »

Two new extended logging bundles have been uploaded.

-Nick
foggy
Veeam Software
Posts: 21138
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by foggy »

Current workaround is increasing the max number of virtual copies using the Hp3PARTempVirtualCopyMaxNumber registry value (current default is 10).
SyNtAxx
Expert
Posts: 149
Liked: 15 times
Joined: Jan 02, 2015 7:12 pm
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by SyNtAxx »

can you give the exact location of this key or where it should be created?

Nick
foggy
Veeam Software
Posts: 21138
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by foggy »

It should be created in the standard Veeam registry hive on the backup server (HKLM\SOFTWARE\Veeam\Veeam Backup and Replication).
SyNtAxx
Expert
Posts: 149
Liked: 15 times
Joined: Jan 02, 2015 7:12 pm
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by SyNtAxx »

ok, I'll try this. Is there an actual ceiling on the number of snapshots or upper bounds?
SyNtAxx
Expert
Posts: 149
Liked: 15 times
Joined: Jan 02, 2015 7:12 pm
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by SyNtAxx »

foggy wrote:Current workaround is increasing the max number of virtual copies using the Hp3PARTempVirtualCopyMaxNumber registry value (current default is 10).

What type of key is this also ?
SyNtAxx
Expert
Posts: 149
Liked: 15 times
Joined: Jan 02, 2015 7:12 pm
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by SyNtAxx »

NM, I created a 32bit DWORD and seet to 20 max snaphots. Veeam accepted the new value as indicated by a new updated warning in my logs. I just updated to 40 snapshots and Hopefully that is enough to get me through until a permanent fix is created.

-Nick
foggy
Veeam Software
Posts: 21138
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by foggy »

Right, it is DWORD. We will raise the default in the next update.
SyNtAxx
Expert
Posts: 149
Liked: 15 times
Joined: Jan 02, 2015 7:12 pm
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by SyNtAxx »

I've upped the maximum snapshots to be 40 and the warnings/errors still persist.

-Nick
foggy
Veeam Software
Posts: 21138
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by foggy »

In other similar case I've heard about it was upped to 100 and this helped. So try to increase it more.
SyNtAxx
Expert
Posts: 149
Liked: 15 times
Joined: Jan 02, 2015 7:12 pm
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by SyNtAxx »

I upped the max snaphots key to 100 and I still get the warning/errors. Whats next, 200 ?

-nick
Aumatics
Novice
Posts: 5
Liked: never
Joined: Oct 14, 2016 11:59 am
Full Name: Aumatics
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by Aumatics »

I'm about to update to update 3, but saw this post. Does this problem also occur on Hyper-V or only on VMware ?
foggy
Veeam Software
Posts: 21138
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by foggy »

SyNtAxx wrote:I upped the max snaphots key to 100 and I still get the warning/errors. Whats next, 200 ?
Depends on how many snapshots do you have... But I'd check with support first.
foggy
Veeam Software
Posts: 21138
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by foggy »

Aumatics wrote:I'm about to update to update 3, but saw this post. Does this problem also occur on Hyper-V or only on VMware ?
Hyper-V integrates with storage snapshots natively, so this functionality is for VMware only.
SyNtAxx
Expert
Posts: 149
Liked: 15 times
Joined: Jan 02, 2015 7:12 pm
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by SyNtAxx »

SyNtAxx wrote:I upped the max snaphots key to 100 and I still get the warning/errors. Whats next, 200 ?

-nick
I still get errors and warnings with setting @ 200 snapshots. There is nowhere close to that many shapshots even on the SAN.

I respectfully ask for the naming convention be rolled back to before Update 3 or provide me a custom .dll or registry key to choose my snapshot name prefix. I've had to babysit my backups all day long since applying update 3. Unacceptable.

Thank you,

NIck
foggy
Veeam Software
Posts: 21138
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by foggy »

Hi Nick, I understand your frustration, further investigation is required. Naming convention was changed with the purpose of distinguishing Veeam auxiliary entities (snapshots, iSCSI hosts) created by the product and those created by customers with the testing or other purposes. The latter were commonly named Veeam_..., which could result in unintended deletion by Veeam B&R.
cdeal
Novice
Posts: 4
Liked: 1 time
Joined: Dec 18, 2012 4:14 pm
Full Name: Craig Deal
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by cdeal » 1 person likes this post

Are there any updates on this issue? I assume that it's recommended not to install update 3 if you use 3PAR snapshot integration?
omegagx
Enthusiast
Posts: 68
Liked: 4 times
Joined: May 09, 2017 6:33 pm
Full Name: Mike G
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by omegagx »

We use 3PAR as well and are waiting on this issue being fixed before we upgrade to Update 3. Any news on this? Is there going to be a knowledge base article describing the issue in detail?
foggy
Veeam Software
Posts: 21138
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by foggy »

Craig, Michael, please contact technical support. They will provide the hotfix after confirming the issue in your environment.
SyNtAxx
Expert
Posts: 149
Liked: 15 times
Joined: Jan 02, 2015 7:12 pm
Contact:

Re: Update3 and 3par SAN snapshots being named incorrectly

Post by SyNtAxx » 1 person likes this post

All,

I belive my issue has been resolved after the application of the custom hotfix and the registry key for Max snapshots. In my case i needed to set max snapshots to 200 in order to not run into issues. The hotfix resolved the mutex timeout issues on creating/removing the snapshots. I wanted to wait on a few days of solid testing before declaring this resolved. I believe in my case the problems have been resolved.

NIck
Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 111 guests