Comprehensive data protection for all workloads
fdietze
Service Provider
Posts: 9
Liked: 4 times
Joined: Jun 22, 2018 7:28 pm
Full Name: Flavio Dietze
Location: Brazil
Contact:

Data Domain 3300 with DD OS 7.5

Post by fdietze »

Hello!

i have a customar that upgraded by himself his data domain 3300 from DD OS 7.1 to DD OS 7.5 (recently launched 23/02/2021).

this customer is already with veeam 11 before this data domain upgrade that was yesterday.

yes i know, the doc says "Dell EMC Data Domain (DD OS version 6.1 to 7.3) with DDBoost license. Both Ethernet and Fibre Channel (FC) connectivity is supported." on https://helpcenter.veeam.com/docs/backu ... 110#target

after upgrade i have the error:
"Error Failed to save Backup Repository: Failed to call RPC function 'DDBoostFindUnitInfo': Failed to connect to storage. Storage: 172.26.5.1, User: veeam. the user has insufficient access rights. Err: 5075."

yes i saw the thread veeam-backup-replication-f2/veeam-fails ... 57064.html and the thread veeam-backup-replication-f2/adding-data ... 57862.html

its not a user situation since i teste everything, i tryied new storage unit, new user, changed clients acceptance and every single option, i compared with other customer running 7.1 and 7.3 and everything looks the same. really appear to be something changed in the ddboost client or connection.

yes, i opened a case to see if has any workaround because data domain os does not offer downgrade or rollback (you need to reinstall from scratch and lost eveything). any comment or workaround this time until full support will be appreciated.
Case #04689950

thanks in advance!
Flávio Dietze
Veeam enthusiast since v4
Veeam Certified Engineer
Veeam Certified Architect
-----------------------------------
Veeam Gold Partner
Veeam Service Provider Gold Partner
Veeam Accredited Service Partner
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by foggy »

Hi Flavio, that's why we don't claim support for a particular version before running tests internally. So far we haven't performed any verification of 7.5 so cannot comment. Have you also contacted Dell EMC for any information on a possible related change in the latest version?
fdietze
Service Provider
Posts: 9
Liked: 4 times
Joined: Jun 22, 2018 7:28 pm
Full Name: Flavio Dietze
Location: Brazil
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by fdietze »

Foggy,

yes i have contact because was emc support from india that upgraded DD without asking nothing to customer. the DD3300 of the customer was with problem in the ssd cache. Support made the upgrade for resolve the ssd, the upgrade did not solve and they exchanged the disk but all the backup problem came on table.

they didnt know anything related to say, their work was pretty amateur and now i have this situation and asked for help in the case number mentioned.

any help appreciated.
Flávio Dietze
Veeam enthusiast since v4
Veeam Certified Engineer
Veeam Certified Architect
-----------------------------------
Veeam Gold Partner
Veeam Service Provider Gold Partner
Veeam Accredited Service Partner
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by foggy »

I'd ask for the case escalation inside Dell EMC support org then. We will definitely do our best to find what've caused the issue, but since it is likely the change on the DDOS side, I'd continue working with Dell EMC as well.
fdietze
Service Provider
Posts: 9
Liked: 4 times
Joined: Jun 22, 2018 7:28 pm
Full Name: Flavio Dietze
Location: Brazil
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by fdietze »

Foggy,

i know, i already raised and talked to escalation but first they asked for sorry after upgrading without asking and then checked ddboost cnnectivity with the ddboot conectivity check and all pass. they said now the problem is at client side (ddboost client inside veeam).

i´m lost.
Flávio Dietze
Veeam enthusiast since v4
Veeam Certified Engineer
Veeam Certified Architect
-----------------------------------
Veeam Gold Partner
Veeam Service Provider Gold Partner
Veeam Accredited Service Partner
fdietze
Service Provider
Posts: 9
Liked: 4 times
Joined: Jun 22, 2018 7:28 pm
Full Name: Flavio Dietze
Location: Brazil
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by fdietze »

Just to let know that tests trought ddbostfs version 7.5 in the os where the console resides can send ddboost traffic do datadomain.
Flávio Dietze
Veeam enthusiast since v4
Veeam Certified Engineer
Veeam Certified Architect
-----------------------------------
Veeam Gold Partner
Veeam Service Provider Gold Partner
Veeam Accredited Service Partner
soncscy
Veteran
Posts: 643
Liked: 312 times
Joined: Aug 04, 2019 2:57 pm
Full Name: Harvey
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by soncscy »

Surely they must have logging that explains exactly what access is not allowed?

AFAIK for any application, not just Veeam, you use a library to connect to the data domains, you don't get to customize this client, you just use the ddboostlib.dll library provided and that's it. It's their own native client returning this, and they should have logging which says specifically which permission was wrong and why or why the access was denied.

I know it can be tough with some support teams, but you need to be persistent and direct. It's non-sense to suggest that an application could some how force and produce a DDboost access error...

As a workaround, couldn't you expose the share temporarily as CIFS and remap the backups? (disable the DDBoost repo in Veeam)
fdietze
Service Provider
Posts: 9
Liked: 4 times
Joined: Jun 22, 2018 7:28 pm
Full Name: Flavio Dietze
Location: Brazil
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by fdietze »

Hello, i agree with soncsy. doesnt make sense for me at all the point of the problem is in veeam but i was needed to ask for something in the case. because veeam could be verifying or analysing in the dev labs the dd os 7.5 and could be a know issue even in dd.

im true case with Dell because the problem is in dd.

after the test ok with ddboostfs, was too little tests. i made other tests without veeam, with just accessing cifs and ddboostfs. initial testes ok and after some moment the dd start to denied read and write in storage units.
now the case is escalated with dell and have two dd engineers looking, one from europe and one from usa.

even with dd os in 7.5 version out of the veeam matrix i was not with confidence that the problem was in veeam.

when dell confirm something and tell something i will update here.
Flávio Dietze
Veeam enthusiast since v4
Veeam Certified Engineer
Veeam Certified Architect
-----------------------------------
Veeam Gold Partner
Veeam Service Provider Gold Partner
Veeam Accredited Service Partner
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by foggy »

We will investigate on our side, but since it is a communication issue between two Dell EMC components, involving them is the right step. Could you please share the Dell case number? I can also say that you're not the only one - we already have other similar cases of imprudent upgrades followed by the same error.
fdietze
Service Provider
Posts: 9
Liked: 4 times
Joined: Jun 22, 2018 7:28 pm
Full Name: Flavio Dietze
Location: Brazil
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by fdietze » 3 people like this post

hello foggy, news and solved!

for sure the problem was at dd side, i already closed veeam case.

the engineer from europe inditified the probleam and made a new patch for dd version 7.5.0.006-680172.rpm even with number lower than dd (7.5.0.5) this patch solved my case and in be inside the next release 7.5.0.10. they reported internally as know issue with dd and will include in the next release notes.
Flávio Dietze
Veeam enthusiast since v4
Veeam Certified Engineer
Veeam Certified Architect
-----------------------------------
Veeam Gold Partner
Veeam Service Provider Gold Partner
Veeam Accredited Service Partner
adurelli
Novice
Posts: 9
Liked: never
Joined: Dec 17, 2019 1:58 pm
Full Name: Anderson Stano Durelli
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by adurelli »

Hi fdietze!

I´m having the same problem as you. Can you send me the patch, so I can test here too?

Tks!
Andreas Neufert
VP, Product Management
Posts: 6707
Liked: 1401 times
Joined: May 04, 2011 8:36 am
Full Name: Andreas Neufert
Location: Germany
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by Andreas Neufert » 1 person likes this post

Please do not share patches outside of support cases. It is important to know which customer is on what patch so that we can follow up if we face issues at updates or overall.
fdietze
Service Provider
Posts: 9
Liked: 4 times
Joined: Jun 22, 2018 7:28 pm
Full Name: Flavio Dietze
Location: Brazil
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by fdietze »

Andreas, the patch i am saying is a patch for Dell EMC Data Domain...
Flávio Dietze
Veeam enthusiast since v4
Veeam Certified Engineer
Veeam Certified Architect
-----------------------------------
Veeam Gold Partner
Veeam Service Provider Gold Partner
Veeam Accredited Service Partner
fdietze
Service Provider
Posts: 9
Liked: 4 times
Joined: Jun 22, 2018 7:28 pm
Full Name: Flavio Dietze
Location: Brazil
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by fdietze »

adurelli, sent you dell case number so they can provide you the patch.
Flávio Dietze
Veeam enthusiast since v4
Veeam Certified Engineer
Veeam Certified Architect
-----------------------------------
Veeam Gold Partner
Veeam Service Provider Gold Partner
Veeam Accredited Service Partner
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by foggy »

Flavio, thanks for the update and please also PM me the case number for reference as I'd like to get the patch for our internal DDOS testing as well.
stevekarra
Technology Partner
Posts: 42
Liked: 6 times
Joined: May 02, 2019 9:19 pm
Full Name: Steve Karra
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by stevekarra » 1 person likes this post

It's a good practice to adhere to the target code revisions to minimize the chances of disruption. 7.2.0.50 is the current stable and well-tested version.

As long as you have a Dell support contract this information can be obtained from https://dl.dell.com/content/docu64492_D ... uage=en_US
Trelor
Enthusiast
Posts: 47
Liked: 15 times
Joined: Apr 27, 2015 6:02 pm
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by Trelor »

Not trying to post jack, but please note that there is also a bug in DDOS 6.2 -> 7.4 which we are facing and DD support has advised to go to 7.5.0.5 due to bug 213735 where a BTree causes MTree issues and file system crashes.
stevekarra
Technology Partner
Posts: 42
Liked: 6 times
Joined: May 02, 2019 9:19 pm
Full Name: Steve Karra
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by stevekarra »

That's unfortunate. You're running out of runway with DDOS 7.5.0.5 - hopefully, all will be good.

DDOS cannot be downgraded which is why I like to follow the target code recommendations. It also gives you breathing room in case you do run into a problem fixed in a later version.
oan
Novice
Posts: 3
Liked: never
Joined: Sep 28, 2011 5:47 am
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by oan »

fdietze wrote: Mar 11, 2021 6:53 pm the engineer from europe inditified the probleam and made a new patch for dd version 7.5.0.006-680172.rpm even with number lower than dd (7.5.0.5) this patch solved my case and in be inside the next release 7.5.0.10. they reported internally as know issue with dd and will include in the next release notes.
Hi, do you have details on the patch? I am sharing your problem and would like to share the solution. :)
Oliver
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by foggy »

Hi Oliver, please contact Dell EMC support to confirm the issue and get the patch.
oan
Novice
Posts: 3
Liked: never
Joined: Sep 28, 2011 5:47 am
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by oan »

Update: Dell EMC has released DDOS 7.5.0.10 which solves the issue.
DRuben
Lurker
Posts: 1
Liked: never
Joined: Mar 04, 2021 1:25 pm
Full Name: Ruben Gerrits
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by DRuben »

a customer of mine has upgraded to 7.5.0.10 and has the same issue
Trelor
Enthusiast
Posts: 47
Liked: 15 times
Joined: Apr 27, 2015 6:02 pm
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by Trelor »

foggy wrote: Mar 16, 2021 12:59 pm Hi Oliver, please contact Dell EMC support to confirm the issue and get the patch.
This is the link to DDOS 7.5.0.10, you need to login first: https://dl.dell.com/content/docu103371

Also from the release notes: DDOS-92082 - Backup applications using DD Boost versions prior to 7.1 may not be able to complete operations like backups, restore, etc
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by foggy »

DRuben wrote: Mar 16, 2021 2:09 pm a customer of mine has upgraded to 7.5.0.10 and has the same issue
Hi Ruben, do you have a case open with either Dell EMC or Veeam regarding that?
JTT
Service Provider
Posts: 99
Liked: 2 times
Joined: Jan 02, 2017 7:31 am
Full Name: JTT
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by JTT »

foggy wrote: Mar 10, 2021 11:41 am Hi Flavio, that's why we don't claim support for a particular version before running tests internally. So far we haven't performed any verification of 7.5 so cannot comment. Have you also contacted Dell EMC for any information on a possible related change in the latest version?
Offtopic, but when will Veeam support it then?? Next VBR version isnt a acceptable solution,
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by foggy »

We have planned the tests but ETA will depend on whether any changes are required on our side to support it - in this case, it inevitably will be one of the next Veeam B&R updates (not necessarily a major version, if that is your concern).
JTT
Service Provider
Posts: 99
Liked: 2 times
Joined: Jan 02, 2017 7:31 am
Full Name: JTT
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by JTT »

Well my only concern is that, how can Veeam claim itself to be a enterprise product, but at the same time cant keep up with the hardware vendors software updates!?
Andreas Neufert
VP, Product Management
Posts: 6707
Liked: 1401 times
Joined: May 04, 2011 8:36 am
Full Name: Andreas Neufert
Location: Germany
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by Andreas Neufert »

I would argue that the opposit is the case. We carefully testing new versions deeply within our QA so that our user install base can trust in our code and reliability. This means we have less code releases than the vendors (Dell releases new major DDOS versions every quarter now and there are updates and patches as well in between) available for the testing and necessary code changes. As well we need to find the right balance with the SDK/DDOS version support as newer SDK means as well that we cap exisiting customers that run on older DDOS versions. Overall our new version adaption process worked reliably in the last 8 years with DataDomain.

To compare this with VMware vSphere Version releases. We support new versions there usually within 90 days of the GA. This is needed because of the massive API and code changes VMware introduces. Even last minute changes in the APIs.

DDOS 7.5.0.10 was released March 16th, 2021
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by foggy »

FYI, we have completed v11 verification against DDOS 7.6 and now all the latest DDOS versions (7.4/7.5/7.6) are officially supported. We will update the documentation to reflect that fact shortly. Thanks!
JTT
Service Provider
Posts: 99
Liked: 2 times
Joined: Jan 02, 2017 7:31 am
Full Name: JTT
Contact:

Re: Data Domain 3300 with DD OS 7.5

Post by JTT »

Veeam support said: Veeam doesn't have any limitations in the documentation about the Firmware,
This means all the versions are supported.


Is this true, so we can apply latest update?
Post Reply

Who is online

Users browsing this forum: Amazon [Bot] and 221 guests