VMs Backup fails due to Trend Micro, Case ID# 02472655

Discussions specific to Microsoft Hyper-V hypervisor

Re: VMs Backup fails due to Trend Micro, Case ID# 02472655

Veeam Logoby stack111 » Mon Feb 05, 2018 6:49 am

This VM is a DC with DHCP and DNS.

However the issue is not for this VM, any VM with trend micro loaded, Veeam cannot do a backup for it, Veeam support know this.

all Windows VMs and Hyper-V hosts are Windows 2012 R2 .

VSS errors in event viewer in guest Vms and hosts wil be shared here today later inshallah.
stack111
Enthusiast
 
Posts: 27
Liked: 1 time
Joined: Thu Jan 18, 2018 8:15 am
Full Name: Ahmad Jayyusi

Re: VMs Backup fails due to Trend Micro, Case ID# 02472655

Veeam Logoby stack111 » Mon Feb 05, 2018 3:51 pm

Hi,

Please find below findings from today remote session with Veeam support:

1.Trend Micro is loaded (services running) into guest VM.
2.When the shadow copy is taken from the host (by Veeam backup), there is a checkpoint file created next to the VM disk; this checkpoint file is then connected to the guest VM and we see it its disk management
3.After the disk is connected, it got to be brought online and mounted so that backup started to take backup from the VM original VHDx file.
4.However, the disk is not brought online, it seems AV prevents disk from being mounted and online, it could be AV is trying to scan the new disk when it tries to come online or trying to scan the boot sector or something. The disk management as a result becomes unresponsive.
5.If we unload Trend Micro (services stopped), the disk created in step 2 above when backup starts is brought online successfully and backup finishes successfully.
6.Per Veeam support, they isolated the issue and found out what can be presented to Trend Micro support.

Maybe there is some options in Trend Micro that you can tune to prevent Trend Micro from accessing the newly created checkpoint (disk) or something..
stack111
Enthusiast
 
Posts: 27
Liked: 1 time
Joined: Thu Jan 18, 2018 8:15 am
Full Name: Ahmad Jayyusi

Re: VMs Backup fails due to Trend Micro, Case ID# 02472655

Veeam Logoby Mike Resseler » Tue Feb 06, 2018 6:11 am

Well,

I'm glad that they could find the issue.

I am assuming now that you need to exclude the following directories:

C:\ProgramData\Microsoft\Windows\Hyper-V\Snapshots
"\Device\HardDiskVolumeShadowCopy*"

And basically any location where your original disks reside

(Did you excluded all typical hyper-v exclusions?)
Mike Resseler
Veeam Software
 
Posts: 4225
Liked: 471 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: VMs Backup fails due to Trend Micro, Case ID# 02472655

Veeam Logoby stack111 » Tue Feb 06, 2018 6:29 am

Hi, we did configure all exclusions for Hyper-V and Veeam, also "\Device\HardDiskVolumeShadowCopy*" and
C:\ProgramData\Microsoft\Windows\Hyper-V\Snapshots also excluded.
stack111
Enthusiast
 
Posts: 27
Liked: 1 time
Joined: Thu Jan 18, 2018 8:15 am
Full Name: Ahmad Jayyusi

Re: VMs Backup fails due to Trend Micro, Case ID# 02472655

Veeam Logoby Mike Resseler » Tue Feb 06, 2018 6:38 am

Then I would talk to Trend Micro again and explain what our support has seen. Hopefully they can tell you what needs to be excluded
Mike Resseler
Veeam Software
 
Posts: 4225
Liked: 471 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: VMs Backup fails due to Trend Micro, Case ID# 02472655

Veeam Logoby stack111 » Tue Feb 06, 2018 6:41 am

Yes we did already communicated with our customer and ask them to communicate with their Trend Micro partner to look into this issue
stack111
Enthusiast
 
Posts: 27
Liked: 1 time
Joined: Thu Jan 18, 2018 8:15 am
Full Name: Ahmad Jayyusi

Re: VMs Backup fails due to Trend Micro, Case ID# 02472655

Veeam Logoby Mike Resseler » Tue Feb 06, 2018 6:44 am

Perfect.

Please keep us informed here also. We are also interested to learn the solution

Many thanks
Mike
Mike Resseler
Veeam Software
 
Posts: 4225
Liked: 471 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: VMs Backup fails due to Trend Micro, Case ID# 02472655

Veeam Logoby Rumple » Fri Feb 09, 2018 3:47 pm

out of curiosity is this trend office scan or is this the new trend Next Gen client that is giving all the grief? I have an enterprise client trying to use NGen on their windows 10 machines and running into all kinds of oddities, like pulse VPN installation failing (but an upgrade of an existing vpn driver works),etc....officescan was always pretty good and stable, but I'm leery of this Ngen client...
Rumple
Enthusiast
 
Posts: 51
Liked: never
Joined: Wed Mar 10, 2010 7:50 pm
Full Name: Mark Hodges

Re: VMs Backup fails due to Trend Micro, Case ID# 02472655

Veeam Logoby stack111 » Fri Feb 09, 2018 4:43 pm

This is trend micro office scan.

Today Trend Micro support asked to disable the below options on the antivirus client on one of the guest VMs and test backup, but backup still failed after disabling them as the disk is not coming online in the guest VM

Scan Network Drive
Scan The Boot Sector Of The USB Storage Device After Plugging In
Scan All Files In Removable Storage Devices After Plugging In
stack111
Enthusiast
 
Posts: 27
Liked: 1 time
Joined: Thu Jan 18, 2018 8:15 am
Full Name: Ahmad Jayyusi

Re: VMs Backup fails due to Trend Micro, Case ID# 02472655

Veeam Logoby Mike Resseler » Mon Feb 12, 2018 7:27 am

These seem like some options that really don't matter imho.

Was this only done at the guest VM or also at the host VM?
Mike Resseler
Veeam Software
 
Posts: 4225
Liked: 471 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: VMs Backup fails due to Trend Micro, Case ID# 02472655

Veeam Logoby stack111 » Mon Feb 12, 2018 7:38 am

Currently at the host there is no Trend Micro agent, only in guest VMs.

I received the below today from Trend Micro support:

We still need to isolate the issue. OfficeScan have different features and for us to isolate what feature is causing the issue we need to do trial and error by disabling each of the components.

Please provide me your availability so that we can perform this isolation. Once we have identified what feature is causing the issue then we can look for a solution or escalate the case to our Product Developers.
stack111
Enthusiast
 
Posts: 27
Liked: 1 time
Joined: Thu Jan 18, 2018 8:15 am
Full Name: Ahmad Jayyusi

Re: VMs Backup fails due to Trend Micro, Case ID# 02472655

Veeam Logoby Mike Resseler » Mon Feb 12, 2018 7:44 am

That's good news in my book. That means they want to investigate what is going on. Keep us informed!
Mike Resseler
Veeam Software
 
Posts: 4225
Liked: 471 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: VMs Backup fails due to Trend Micro, Case ID# 02472655

Veeam Logoby nmdange » Mon Feb 12, 2018 3:06 pm

Check if Trend Micro disabled volume automount. You can check this by opening a command prompt, typing "diskpart", and then "automount". If automount is disabled, you can enable it with diskpart by doing "automount enabled".
nmdange
Expert
 
Posts: 303
Liked: 73 times
Joined: Thu Aug 20, 2015 9:30 pm

Re: VMs Backup fails due to Trend Micro, Case ID# 02472655

Veeam Logoby stack111 » Mon Feb 12, 2018 3:24 pm

I ran below commands

C:\Windows\system32>diskpart

Microsoft DiskPart version 6.3.9600

Copyright (C) 1999-2013 Microsoft Corporation.
On computer: xxxx

DISKPART> automount

Automatic mounting of new volumes enabled.

DISKPART> automount enable

Automatic mounting of new volumes enabled.

DISKPART>

Then retried backup but same issue..
stack111
Enthusiast
 
Posts: 27
Liked: 1 time
Joined: Thu Jan 18, 2018 8:15 am
Full Name: Ahmad Jayyusi

Re: VMs Backup fails due to Trend Micro, Case ID# 02472655

Veeam Logoby stack111 » Tue Feb 13, 2018 8:35 am

Problem is solved; backup is working now for guest VMs.

The issue was that \Device\HarddiskVolume* was not excluded, once excluded in Trend Micro, backup is working!

I think Veeam should update their needed antivirus exclusion list in this article: https://www.veeam.com/kb1999

We found that \Device\HarddiskVolume* should be excluded from this article: https://kc.mcafee.com/corporate/index?p ... id=KB54457
stack111
Enthusiast
 
Posts: 27
Liked: 1 time
Joined: Thu Jan 18, 2018 8:15 am
Full Name: Ahmad Jayyusi

PreviousNext

Return to Microsoft Hyper-V



Who is online

Users browsing this forum: No registered users and 6 guests