VBR 9.5 U3 and 3PAR 3.3.1 EMU1

[nathano]

I'll be updating to U3 for 9.5 next week and then early next year I hope to have our 3PARs upgraded to 3.3.1 EMU1.

Are there any known issues with 9.5U3 and 3par 3.3.1 EMU1 ?

Thanks
Nathan

[jazzoberoi]

+ Would be doing the same and would like to know of any known issues.

[DGrinev]

Hi,

There are no issues expected, since the QA team told me they have tested Update 3 in the internal lab with the 3PAR 3.3.1. Thanks!

[nathano]

Did the update to 3.3.1.269 (MU1)+P09,P11,P18,P21,P24 last night and now the storage integration is not working. Didn't have the fail over option selected in the jobs, have set that now and running the jobs and they are failing over to standard backup.

I'll call support tomorrow, it's late

[nathano]

I have logged case 02603838 with support about this one.

[nathano]

So here's the current state of things - 3par integration still not working..

VBR 9.5 U3
3par has been upgraded to 3.3.1.269 (MU1)+P09,P11,P18,P21,P24 which is what caused the problem

support has identified this part of the logs

Code: Select all

10.02.2018 21:58:01] <12> Info     [CLI] Connecting to HPE 3PAR '192.168.101.43'
[10.02.2018 21:58:01] <12> Info     [Ssh] Creating new connection 516e971b-232e-4890-957f-f43e70e2a0f3 - [host: '192.168.101.43', port: 22, user: 'xxx', elevation to root: 'no', autoSudo: no, auth type: 'Password', host name: , IPs: [192.168.101.43].].
[10.02.2018 21:58:01] <12> Info     [Ssh] logon, host: '192.168.101.43', port: 22, user: 'xxx', elevation to root: 'no', autoSudo: no, auth type: 'Password', host name: , IPs: [192.168.101.43].
[10.02.2018 21:58:01] <12> Info     [Ssh] Connected to 192.168.101.43 while other addresses () are unavailable.
[10.02.2018 21:58:01] <12> Warning  Failed to create SSH connection to host: '192.168.101.43', port: 22, user: 'xxx', elevation to root: 'no', autoSudo: no, auth type: 'Password', host name: , IPs: [192.168.101.43].. Server does not support diffie-hellman-group1-sha1 for keyexchange
[10.02.2018 21:58:01] <12> Warning  Failed to create Granados SSH connection, switch to Renci SSH. Server does not support diffie-hellman-group1-sha1 for keyexchange

So from reading the log file it would appear as though VBR is attempting to connect to the 3par with diffie-hellman-group1-sha1 for keyexchange!

As per page 16 of the 3par 3.3.1 release notes https://support.hpe.com/hpsc/doc/public ... 17203en_us it states

Supported Ciphers
• KexAlgorithms: diffie-hellman-group-exchange-sha256
• Ciphers: chacha20-poly1305@openssh.com, aes256gcm@openssh.com, aes128-
gcm@openssh.com, aes256-ctr,aes192-ctr, and aes128-ctr.
• MACs: hmac-sha2-512-etm@openssh.com, hmac-sha2-256-etm@openssh.com, hmacripemd160-etm@openssh.com,
umac-128-etm@openssh.com, hmac-sha2-512, hmacsha2-256,
hmac-ripemd160, and umac-128@openssh.com.

Previously supported Ciphers
• KexAlgorithms: curve25519-sha256@libssh.org, ecdh-sha2-nistp256, ecdh-sha2-
nistp384, ecdh-sha2-nistp521, diffie-hellman-group-exchange-sha256, diffiehellman-group-exchange-sha1,
diffie-hellman-group14-sha1, diffie-hellmangroup1-sha1
• Ciphers: aes192-ctr, aes256-ctr, aes128-ctr, aes192-cbc, aes256-cbc, aes128-cbc,
3des-cbc
• MACs: hmac-sha1 and hmac-sha1-96

Reading the release notes it would seem as though the ONLY support Kex is now diffie-hellman-group-exchange-sha256

Support are wanting me to confirm that the 3par is running diffie-hellman-group-exchange-sha256 .

When I connect to the 3par from an SSH client this is what I am shown.


So to me that looks like sha256 is being used.

If anyone has any ideas I'd love to hear them

Thanks
Nathan

[foggy]

Hi Nathan, SSH warning is not the case here, it is just a failover to another library: "Failed to create Granados SSH connection, switch to Renci SSH". The actual issue here (according to your logs) is WSAPI connection. You can try to restart WSAPI service (stopwsapi -> startwsapi) and check the number of WSAPI connections on 3PAR (showwsapisession). Further investigation could require involving HPE support.

[nathano]

Thanks, I know the Veeam suppot guy checked the WSAPI on the 3par and stopped/started it. I just logged in and ran the showwsapisession, there are no WSAPI sessions.

[jazzoberoi]

Will be following this thread as we've got a 3Par that i havent upgraded to 3.3.1 as of yet

[nathano]

I was going to come back and tag you (if possible) jazzoberoi and see if you had done the upgrade yet. clearly I'm not doing the other 3par just yet..

right now I'm installed a fresh VBR on a test machine and will try and connect that to the 3par. work out if it's 3par or something screwy with my VBR. one theory I have is that the old ssh cert from the 3par is still in place on the vbr and that's why it's failing to connect now.. of the new VBR connects then I'll check certs on the VBR, if the new VBR doesn't connect then it's got to be on the 3par/3par integration side of things.

[nathano]

test with a new VBR done and it does the same thing.

on both existing and test VBR it get's to this screen after entering the credentials.



which means it's talking to the 3par to determine what connectivity we have. then it fails after this step. I'm guessing something in the 3par integration needs tweaking, possibly due to patches that have been applied that weren't tested by Veeam? just guessing

[foggy]

As I've mentioned, this is not related to SSH, which finally succeeds, after failing over to using another library. It seems like it worked for some time after upgrading to U3, so might be indeed some HPE patches - do you know when they were applied?

[nathano]

This issue has now been resolved. I was on a call with another Veeam Support person and doing some more testing.. and he then found another similar case..

solution = create a new user on the 3par and use that account for Veeam to connect with!

[DGrinev]

Nathan,

Thank you for following up with the solution, it can be helpful for further readers!

[nathano]

no problem at all. that's what I posted all the details here, hopefully it can help someone else.

just very strange

[nathano]

Just had a call from the 3par team, all 3.3.1 upgrades are on hold. he wouldn't give me any details as to why but I'm guessing it's something to do with the upgrade process itself as I asked about the system that's already been upgraded to 3.3.1 and he said that's fine - hence my assumption it's the upgrade process.

so we are going up to MU6 and will wait there until 3.3.1 is back on the cards again.