Veeam constantly creating/destroying LUNs on NetApp

Availability for the Always-On Enterprise

Veeam constantly creating/destroying LUNs on NetApp

Veeam Logoby lightsout » Mon Sep 26, 2016 7:57 pm 1 person likes this post

Infrastructure is a Veeam v9 U2 server running on Windows 2012 R2 connecting to a 7-mode FAS3220 running 8.2.4P4.

I put a ticket in with NetApp due to performance issues, and they believe it is due to Veeam constantly creating and destroying LUNs. I've added the NetApp servers to my Veeam infrastructure so I can do restores from snapshots, etc. Between 08:30 and 16:00 they saw 109 LUNs being created and destroyed. Is this normal?

Example logs from my NetApp below. It even seems to be scanning the same LUN!

Mon Sep 19 10:39:05 EDT [NetApp01:lun.map:info]: LUN /vol/Server2_LUN1/Veeam_Server2_LUN1_Rescan was mapped to initiator group Veeam_veeam-server.domain.com_VeeamInitiator=0
Mon Sep 19 10:39:05 EDT [NetApp01:iscsi.notice:notice]: ISCSI: New session from initiator iqn.2006-06.com.veeam:veeam-server.domain.com at IP addr 10.20.30.40
Mon Sep 19 10:42:36 EDT [NetApp01:lun.map.unmap:info]: LUN /vol/Server2_LUN1/Veeam_Server2_LUN1_Rescan unmapped from initiator group Veeam_veeam-server.domain.com_VeeamInitiator
Mon Sep 19 10:42:38 EDT [NetApp01:lun.offline:warning]: LUN /vol/Server2_LUN1/Veeam_Server2_LUN1_Rescan has been taken offline
Mon Sep 19 10:42:39 EDT [NetApp01:lun.destroy:info]: LUN /vol/Server2_LUN1/Veeam_Server2_LUN1_Rescan destroyed
Mon Sep 19 10:43:11 EDT [NetApp01:lun.map:info]: LUN /vol/Server2_LUN1/Veeam_Server2_LUN1_Rescan was mapped to initiator group Veeam_veeam-server.domain.com_VeeamInitiator=0
Mon Sep 19 10:43:11 EDT [NetApp01:iscsi.notice:notice]: ISCSI: New session from initiator iqn.2006-06.com.veeam:veeam-server.domain.com at IP addr 10.20.30.40
Mon Sep 19 10:46:43 EDT [NetApp01:lun.map.unmap:info]: LUN /vol/Server2_LUN1/Veeam_Server2_LUN1_Rescan unmapped from initiator group Veeam_veeam-server.domain.com_VeeamInitiator
Mon Sep 19 10:46:45 EDT [NetApp01:lun.offline:warning]: LUN /vol/Server2_LUN1/Veeam_Server2_LUN1_Rescan has been taken offline
Mon Sep 19 10:46:46 EDT [NetApp01:lun.destroy:info]: LUN /vol/Server2_LUN1/Veeam_Server2_LUN1_Rescan destroyed
Mon Sep 19 10:47:18 EDT [NetApp01:lun.map:info]: LUN /vol/Server2_LUN1/Veeam_Server2_LUN1_Rescan was mapped to initiator group Veeam_veeam-server.domain.com_VeeamInitiator=0
Mon Sep 19 10:47:18 EDT [NetApp01:iscsi.notice:notice]: ISCSI: New session from initiator iqn.2006-06.com.veeam:veeam-server.domain.com at IP addr 10.20.30.40
Mon Sep 19 10:50:48 EDT [NetApp01:lun.map.unmap:info]: LUN /vol/Server2_LUN1/Veeam_NetApp_LUN1_Rescan unmapped from initiator group Veeam_veeam-server.domain.com_VeeamInitiator
Mon Sep 19 10:50:50 EDT [NetApp01:lun.offline:warning]: LUN /vol/NetApp_LUN1/Veeam_NetApp_LUN1_Rescan has been taken offline
Mon Sep 19 10:50:51 EDT [NetApp01:lun.destroy:info]: LUN /vol/NetApp_LUN1/Veeam_NetApp_LUN1_Rescan destroyed
Mon Sep 19 10:51:23 EDT [NetApp01:lun.map:info]: LUN /vol/NetApp_LUN1/Veeam_NetApp_LUN1_Rescan was mapped to initiator group Veeam_veeam-server.domain.com_VeeamInitiator=0
Mon Sep 19 10:51:23 EDT [NetApp01:iscsi.notice:notice]: ISCSI: New session from initiator iqn.2006-06.com.veeam:veeam-server.domain.com at IP addr 10.20.30.40
Mon Sep 19 10:54:53 EDT [NetApp01:lun.map.unmap:info]: LUN /vol/NetApp_LUN1/Veeam_NetApp_LUN1_Rescan unmapped from initiator group Veeam_veeam-server.domain.com_VeeamInitiator
Mon Sep 19 10:54:55 EDT [NetApp01:lun.offline:warning]: LUN /vol/NetApp_LUN1/Veeam_NetApp_LUN1_Rescan has been taken offline
Mon Sep 19 10:54:56 EDT [NetApp01:lun.destroy:info]: LUN /vol/NetApp_LUN1/Veeam_NetApp_LUN1_Rescan destroyed
Mon Sep 19 10:55:28 EDT [NetApp01:lun.map:info]: LUN /vol/NetApp_LUN1/Veeam_NetApp_LUN1_Rescan was mapped to initiator group Veeam_veeam-server.domain.com_VeeamInitiator=0
Mon Sep 19 10:55:28 EDT [NetApp01:iscsi.notice:notice]: ISCSI: New session from initiator iqn.2006-06.com.veeam:veeam-server.domain.com at IP addr 10.20.30.40
Mon Sep 19 10:58:59 EDT [NetApp01:lun.map.unmap:info]: LUN /vol/NetApp_LUN1/Veeam_NetApp_LUN1_Rescan unmapped from initiator group Veeam_veeam-server.domain.com_VeeamInitiator
Mon Sep 19 10:59:01 EDT [NetApp01:lun.offline:warning]: LUN /vol/NetApp_LUN1/Veeam_NetApp_LUN1_Rescan has been taken offline
Mon Sep 19 10:59:02 EDT [NetApp01:lun.destroy:info]: LUN /vol/NetApp_LUN1/Veeam_NetApp_LUN1_Rescan destroyed
lightsout
Expert
 
Posts: 186
Liked: 47 times
Joined: Thu Apr 10, 2014 4:13 pm

Re: Veeam constantly creating/destroying LUNs on NetApp

Veeam Logoby AFalb » Tue Sep 27, 2016 11:52 am

Hello,
we have absolutely the same behavior here on a FAS3210 running 8.2.4P3, but the performance isn't impacted to greatly.
AFalb
Influencer
 
Posts: 10
Liked: 1 time
Joined: Tue Sep 27, 2016 11:48 am

Re: Veeam constantly creating/destroying LUNs on NetApp

Veeam Logoby lightsout » Tue Sep 27, 2016 12:50 pm

That is good.... I think. I've raised a ticket #01916272. If nothing else, I'd like to scan it less frequently! Not sure much changes in the intervening 2-3 minutes.
lightsout
Expert
 
Posts: 186
Liked: 47 times
Joined: Thu Apr 10, 2014 4:13 pm

Re: Veeam constantly creating/destroying LUNs on NetApp

Veeam Logoby foggy » Tue Sep 27, 2016 4:50 pm

Letting support look into this from Veeam B&R side of things is the right step, indeed.
foggy
Veeam Software
 
Posts: 15087
Liked: 1110 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Veeam constantly creating/destroying LUNs on NetApp

Veeam Logoby AFalb » Mon Oct 03, 2016 6:59 am

You can look into this WhitePaper on Page 8 for some Information:
https://www.veeam.com/wp-netapp-configu ... guide.html
AFalb
Influencer
 
Posts: 10
Liked: 1 time
Joined: Tue Sep 27, 2016 11:48 am

Re: Veeam constantly creating/destroying LUNs on NetApp

Veeam Logoby lightsout » Mon Oct 03, 2016 12:53 pm

Thanks, that is very interesting. I believe I am seeing it below 10 minutes, but I can certainly set the value very high and see what happens!
lightsout
Expert
 
Posts: 186
Liked: 47 times
Joined: Thu Apr 10, 2014 4:13 pm

Re: Veeam constantly creating/destroying LUNs on NetApp

Veeam Logoby lightsout » Tue Oct 04, 2016 2:44 pm

I set the value, no difference, Veeam is still creating and destroying LUNs every few minutes.
lightsout
Expert
 
Posts: 186
Liked: 47 times
Joined: Thu Apr 10, 2014 4:13 pm

Re: Veeam constantly creating/destroying LUNs on NetApp

Veeam Logoby tony.grilley » Thu Oct 27, 2016 3:12 pm 1 person likes this post

Just to update this, the behavior is part of the rescan operation and expected. Increasing the rescan interval has (mostly) corrected this, and NetApp is looking in to why destroying the FlexClone connections is taxing the CPU so much, as those are the LUN's being destroyed.
tony.grilley
Veeam Software
 
Posts: 37
Liked: 11 times
Joined: Mon Apr 21, 2014 2:40 pm
Full Name: Tony Grilley

Re: Veeam constantly creating/destroying LUNs on NetApp

Veeam Logoby MichaelCade » Thu Oct 27, 2016 10:45 pm

It is indeed linked to that Rescan process that Veeam makes on the storage side, increasing the time iteration of this can reduce or remove that impact on that production workload, another thing to consider is making sure that you are only rescanning the relevant vSphere datastore Volumes on the NetApp, you can exclude all other volumes by going to Storage Infrastructure and right click the storage array and properties in there you are able to exclude volumes completely.
Regards,

Michael Cade
Technical Evangelist
Veeam Software
Email: Michael.Cade@Veeam.com
Twitter: @MichaelCade1
MichaelCade
Veeam Software
 
Posts: 63
Liked: 9 times
Joined: Mon Mar 23, 2015 11:55 am
Location: Cambridge, United Kingdom
Full Name: Michael Cade


Return to Veeam Backup & Replication



Who is online

Users browsing this forum: No registered users and 34 guests