NetApp Ontap 9.x Storage Snapshot Support and Issues

[DaStivi]

Hello there,
i'm aware that NetApp Ontap 9 is still in RC1 for now and that Veeam does not support it officially... but I'm in the position to at least have to test the integration.... and it looks quite good, Storage Snapshot integration with NFS VM Stores is working as expected, but i've some issues with Snapshot Integration on FibreChannel (FCP) VMFS Stores!

i've enabled iSCSI Lifs on my FCP SVMs so that my virtual Backup Proxys should be able to connect to that Fibre LUNs, i saw once that Veeam was cloning the volume and trying to map but than failed ...

i saw also some errors on the netapp command line event logs:
iscsi.protocol.violation: ISCSI: iSCSI protocol violation, 'Initiator sent bad opcode 06 during login sequence'

but not getting this error everytime... maybe something changed here in ontap 9...

i can rescan the volumes with the FCP LUNs/Store inside without any issue! UPDATE: i saw that in rescan the LUN was getting cloned shortly... btw. i can create storage snapshots via veeam manually...

the other issue have not to be related to ontap 9, maybe this is common issue... i've opened a support case for this issue too (01842614)

the Settings for the "Failover to Standard Backup" and the "Failover to primary Snapshot" is getting ignored... that means i've not checked the boxes and i can clearly see that when ah storage snapshot is not working (because my iscsi issue above) it is still using hotadd method than....

also i would like to get vm backups from my snapvaulted Storage Snapshot, and i saw in the job logs and with performance logs from primary storage that the backups getting downloaded from primary snapshot! the snapvault updated is still transfering while veeam allready backup machines, the option to failback to primary snapshot is not set....

[DerOest]

Hi DaStivi,

you sound like you know what you're doing, but the last point just caught my eye - I'm not sure if you missed that, or if you think that it's bugged with Ontap 9.

Backup from SnapVault Destination:
Have you checked the checkbox for this?
Edit Job -> Secondary Target -> select the SnapVault -> Edit -> "use as the data source"

[DaStivi]

Backup from SnapVault Destination:
Have you checked the checkbox for this?
Edit Job -> Secondary Target -> select the SnapVault -> Edit -> "use as the data source"

i've of course enabled on "secondary target" - "use ad data source" that is what makes me suspicious, because in job log i clearly see that snapvault transfer is still in progress and veeam is allready downloading vm files from primary snapshot... i'm not totally aware if this is also a ontap 9 issue...

update:
i've tested this on NetApp Release 8.3.2P1 now...
i cann still see that snapvault update is still transfering and the machines are getting started to backup'd from storage snapshot, what means it must be from primary!

update2:
i can see in the Agent.xxxx.Source.hostname.vmdk.log file that its clearly using the primary snapshot.... (ip from primary nfs svm)

and on the netapps it self i can see with sysstat that the primary netapp has huge amount of net out traffic... whereby the secondary (backup) netapp doesn't have any significant net out....

[marco.horstmann]

i've tested this on NetApp Release 8.3.2P1 now...

Please notice that NetApp 8.3.2 ist currently unsupported by Veeam.
https://helpcenter.veeam.com/backup/vsp ... ml#storage

[DaStivi]

yes thanks your support sent me this information... but my best guess is thats not related to ontap 9 or 8.3.2 at least the issue that the options for failback and the use of secondary data source are getting ignored...

maybe someone here with 8.3.1, who is using storage snapshots with snapvault can check if this issue happens in an "supported" state...

[Gostev]

As far as I recall, there were some issues with this checkbox that were fixed early in 9.5 development cycle. I cannot remember all the details since we discussed this a few months ago, but this check box did not always behave in the way customers expected, and we have addressed that. Thanks!

[DaStivi]

ok thanks for that information, i have access to 9.5 beta but won't install it as upgrade from 9.0 to 9.5beta and than further more to rc/final is not supported as per beta information...

[Pat490]

will Ontap 9.1 be officially supported with VBR 9.5 ?

[foggy]

ONTAP 9.0 - yes, ONTAP 9.1 - no (currently it is still in RC stage only).

[Pat490]

ok thank you

[Pat490]

Release notes say that Ontap 9.0 is supported but what about 9.0p1? Are minor patch updates always supported?

https://www.veeam.com/veeam_backup_9_5_ ... _en_rn.pdf (Page 10)
"Data ONTAP versions from 8.1 up to 9.0"

[Gostev]

Yes, normally they are always supported since they are merely bug fixing updates.

[Pat490]

We are just implementing our new NetApp storage system. According to https://helpcenter.veeam.com/backup/vsp ... etapp.html a NetApp license for FlexClone or SnapRestore is now necessary! Why is FlexClone preferred? Does it make any difference in backup handling/performance which license we buy?

We use cDot 8.3.2P7 with FC.

[foggy]

Yes, FlexClone is preferred for better job performance rates.

[Pat490]

what does the following error mean?

Code: Select all

Failed to create snapshot for LUN DS_xyz Details: Failed to restore file from Snapshot copy Veeam_vol_xyz for volume "vol_xyz" on Vserver "SVM_1". Reason: Volume is full or Volume crossed autodelete threshold.

backup of another VM on different Vol/LUN works fine.

[foggy]

Depending on the license, the space for the entire VM might be required during restore operation and seems there's not enough space in this case.

[Pat490]

How can I determine if Veeam is trying to use SnapRestore or FlexClone? I have added test-licenses for both in my NetApp-Cluster.
As far as I understand, FlexClone should need much less space compared to SnapRestore.

[foggy]

The license used to clone the volume (FlexClone и SnapRestore) can be seen in the job log file.

[Pat490]

you mean in Job.<backupjob-name>.Backup.log?

I can not find the word "flexclone" but I see e.g. this:

Code: Select all

[21.12.2016 00:31:11] <01> Error       bei Veeam.Backup.SanPlugin.NetAppLib.CNaHostApiImpl.SnapRestore(LogLevels logLevel, CNaConnection connection, String srcFile, String dstFile, String snapshotName)
[21.12.2016 00:31:11] <01> Error       bei Veeam.Backup.SanPlugin.NetAppLib.CNaVServerClient.SnapRestoreFileAsync(String srcFile, String dstFile, String snapshotName)
[21.12.2016 00:31:11] <01> Error       bei Veeam.Backup.SanPlugin.NetAppLib.CNaClientUtils.SnapRestoreFileSync(INaClient client, String srcFile, String dstFile, String snapshotName)
[21.12.2016 00:31:11] <01> Error       bei Veeam.Backup.SanPlugin.NetApp.CNetAppLunCloneCreator.CreateSnapRestoreCloneForVServer(CNaVServerClient vServerClient, String lunPath, String lunClonePath, String snapshotName)

Does this mean it is only trying SnapRestore and not FlexClone?

[foggy]

Yes, this means SnapRestore is being used. Please review this section for NetApp storage snapshots requirements and limitations.

[Pat490]

thank you very much for clarification!

[Pat490]

I had to do a "Rescan Storage" so that Veeam found the newly added FlexClone license!

Does Veeam rescan storage automatically from time to time?

[foggy]

Yes, automatic discovery that collects this kind of information runs every 7 days.

[DerOest]

ONTAP 9.0 - yes, ONTAP 9.1 - no (currently it is still in RC stage only).

Hi Foggy,

looking at the System Requirements https://helpcenter.veeam.com/docs/backu ... tml?ver=95, Ontap 9.1 is not supported.
Now we've got a FAS2650 as new backup destination (SnapVault target) - but they only ship with Ontap 9.1+
Do you (or anyone else here) know if 9.1 works despite not being officially supported yet?
Or do you know when it will be supported?

Thanks a lot & greetings from Germany!

[marco.horstmann]

Hi DerOest,
I talked the last days to Christian Furchert, NetApp SE from office Hamburg.
We discovered a critical bug inside of Ontap 9.1. Please talk to your local NetApp SE
that can help to understand if this bug will hit you too. He can contact Christian for
more details on this.

While this bug is existing I could imagine we cannot support 9.1 until they
released a patch for this. We forwarded this issue already to our QA.

Kind regards und ein fröhliches Moin Moin aus Norddeutschland
Marco

P.S. It's not a bug in combination with Veeam it seems to be a general problem
with one OS function we use intensively.

[Pat490]

are you allowed to give any more details about what bug it is?

[marco.horstmann]

Hi Patrick,

BfSS using clones in the NetApp system and it seems that one clone function produces timeout for production systems.

I don't know if this happens in this special setup or maybe happens in all systems. NetApp R&D is working on this issue.

Regards
Marco

[Pat490]

BfSS = Backup from Storage Snapshot?
Please post any updates as soon as you hear back from NetApp about this issue!

[marco.horstmann]

BfSS = Backup from Storage Snapshot?
Please post any updates as soon as you hear back from NetApp about this issue!

Yes, Backup from Storage Snapshot

As soon as I hear something new I will update this post.

[DerOest]

Ontap 9.1P1 fixed some issues, not sure if the one Marco talks about is one of them.
Has anyone heard anything regarding this?

https://mysupport.netapp.com/NOW/downlo ... tap/9.1P1/