VEP keep failing + enhancement suggestions

Backup agent for Microsoft Windows servers and workstations (formerly Veeam Endpoint Backup FREE)

VEP keep failing + enhancement suggestions

Veeam Logoby bhwong » Mon Sep 26, 2016 1:32 am

The rate of a successful backup for VEP is very low in my environment, not to mention that in the rare few times when it completed successfully, it took about 2-3 days for it to complete a backup of folders under 2TB, far slower than the default Windows Server Backup, which took less than 24 hours for the same set of folders. Why is this so?

I have a few suggestions to improve this:

1. The performance should improve if VEP can make use of existing persistent shadow copies instead of creating a temp one every time it started it's backup job.

2. Allow configuration of DataMoverLocalFastPath and EndPointBlockSize from the UI instead of editing the registry. Better if there is a slider for administrator to adjust between performance vs reliability to make this more user friendly.

3. Ability to resume from previous failed backup job, instead of starting a recovery point.

4. WAN accelerator and compression will be helpful.
bhwong
Enthusiast
 
Posts: 92
Liked: 2 times
Joined: Thu May 24, 2012 9:57 am
Full Name: Boon Hong Wong

Re: VEP keep failing + enhancement suggestions

Veeam Logoby Dima P. » Mon Sep 26, 2016 2:48 pm

Hi bhwong,

Thanks for the prompt feedback.
successful backup for VEP is very low in my environment, not to mention that in the rare few times when it completed successfully, it took about 2-3 days for it to complete a backup of folders under 2TB

May I ask is that a file level backup or volume level backup? Did you open a support case regarding this issues?
DataMoverLocalFastPath

Connection between VEB components via shared memory is supported out of the box, so I am not sure why you are referencing to this registry value
EndPointBlockSize

Will be available in UI in next version
WAN accelerator and compression will be helpful.

Can you elaborate, need to know your potential use case for WAN accelerator in VEB? Regarding compression – the UI control will become available in next version as well.
Dima P.
Veeam Software
 
Posts: 6698
Liked: 479 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: VEP keep failing + enhancement suggestions

Veeam Logoby bhwong » Tue Sep 27, 2016 1:32 am

It's file level backup as mentioned "same set of folders".

And yes, I have open a support case where support suggested me to add DataMoverLocalFastPath in the registry to improve reliability for backing up over the WAN connection. It does help as I finally manage to get one successful incremental backup. But the failure rate is still too high. So support suggest that I backup to local NAS and then copy the incremental backup to remote NAS. However, the failure rate for local backup is still very high as well.

I don't mind slow performance in exchange for better reliability. However, VEP is both slower and less reliable than our default Windows Server Backup. The main reason why I wanted to give VEP a try is that it support incremental backup that Windows Server Backup do not. I expect VEP will help to reduce the time and WAN bandwidth required to backup our shared folders in our Windows Storage Server over to our offsite storage, but unfortunately, it keep failing.
bhwong
Enthusiast
 
Posts: 92
Liked: 2 times
Joined: Thu May 24, 2012 9:57 am
Full Name: Boon Hong Wong

Re: VEP keep failing + enhancement suggestions

Veeam Logoby Dima P. » Tue Sep 27, 2016 6:11 pm

May I ask why you are using file level backup instead of volume level backup?
Dima P.
Veeam Software
 
Posts: 6698
Liked: 479 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: VEP keep failing + enhancement suggestions

Veeam Logoby bhwong » Wed Sep 28, 2016 12:49 am

Because I just want to make backup of the frequently changed shared folders without the OS, applications, temp files etc that will consume my backup storage, especially when I intend to keep multiple copies.

I need a backup software that can copy locked file and preserve NTFS permission. Most sync applications I tried do not support these. And Windows Server backup do not support incremental backup.

FYI, previously, even Windows Server backup keep failing. The problem was resolved once I remove Windows Storage Server dedup. It's quite shocking that Microsoft failed to verify that their own backup works with their own deduplication before releasing them.

btw, this is a physical server, not a virtual machine. Will it be able to restore to new server using difference hardware and drivers?
bhwong
Enthusiast
 
Posts: 92
Liked: 2 times
Joined: Thu May 24, 2012 9:57 am
Full Name: Boon Hong Wong

Re: VEP keep failing + enhancement suggestions

Veeam Logoby bhwong » Mon Oct 10, 2016 1:31 am

Veeam Support has told me that it's our unreliable WAN that cause offsite backup to fail despite the fact that our Windows Server Backup has done the same successfully and repeatedly over our WAN connection. So I'm not very convinced with this explanation.

FOr the benefit of doubt, I have even deployed a new NAS from another brand (Synology instead of QNAP) and this is connected directly to the same switch as the server we want to backup, so there is no "excuse" of any unreliable network connectivity issue.

Guess what? Veeam backup keep failing too! I have selected the same set of folders (under 2TB) for both VEP and WSB (Windows Server Backup) and WSB has backed up successfully the next day while Veeam was still at 25% despite both started around the same time. I'm puzzled over the extremely slow performance. And after more than 65 hours later, it still failed to complete a simple local backup job.

Why is such a high failure rate and poor performance be possible? Especially it's from a good reputable backup vendor like Veeam? What could be the other possible cause?

Error: Timed out to wait until the size of mount cache will be less than threshold Agent failed to process method {FileBackup.SyncDirs}.
bhwong
Enthusiast
 
Posts: 92
Liked: 2 times
Joined: Thu May 24, 2012 9:57 am
Full Name: Boon Hong Wong

Re: VEP keep failing + enhancement suggestions

Veeam Logoby Dima P. » Mon Oct 10, 2016 4:34 pm

Because I just want to make backup of the frequently changed shared folders without the OS, applications, temp files etc that will consume my backup storage, especially when I intend to keep multiple copies.

Try to run a volume level backup with a system folders being excluded. Due it's nature volume level backup is much faster, specifically when talking about huge amount of data.
Will it be able to restore to new server using difference hardware and drivers?

If you have a recovery media and volume level backup of entire OS, then yes.
Why is such a high failure rate and poor performance be possible?

Unfortunately, support team was right. VEB 1.5 is not designed for large file level backups over slow WAN links (it does not have the required reconnect logic and such). An option to perform a direct backup to the cloud connect repository and backup cache feature are coming in the next version. Both should solve your issue completely.
Dima P.
Veeam Software
 
Posts: 6698
Liked: 479 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: VEP keep failing + enhancement suggestions

Veeam Logoby bhwong » Tue Oct 11, 2016 2:06 am

Thanks! Glad to hear that our feedbacks are noted and I can't wait for the next version! :)

FYI, I have also tried local backup with direct connection and it failed too! This is local 1GB ethernet network, not over the WAN.

But I will give the volume backup excluding system folders a try once VEP has complete the local backup I'm retrying. I didn't know that a volume backup can exclude folders.
bhwong
Enthusiast
 
Posts: 92
Liked: 2 times
Joined: Thu May 24, 2012 9:57 am
Full Name: Boon Hong Wong

Re: VEP keep failing + enhancement suggestions

Veeam Logoby Dima P. » Tue Oct 11, 2016 2:03 pm

I didn't know that a volume backup can exclude folders

This is a bit tricky: choose File level backup mode, select the volume and then uncheck the folders. Label close to the volume should indicate that you are still using volume level backup.
Dima P.
Veeam Software
 
Posts: 6698
Liked: 479 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: VEP keep failing + enhancement suggestions

Veeam Logoby bhwong » Mon Oct 17, 2016 1:22 am

I tried volume level backup and it still failed with this error:

Unable to exclude files from snapshot: Shadow Copy Optimization Writer timed out (see Windows event log for more info).
bhwong
Enthusiast
 
Posts: 92
Liked: 2 times
Joined: Thu May 24, 2012 9:57 am
Full Name: Boon Hong Wong

Re: VEP keep failing + enhancement suggestions

Veeam Logoby bhwong » Mon Oct 17, 2016 3:31 am

I also notice that despite I have selected only 3 folders to backup under volume level backup, it still backed up everything in the whole volume!
bhwong
Enthusiast
 
Posts: 92
Liked: 2 times
Joined: Thu May 24, 2012 9:57 am
Full Name: Boon Hong Wong

Re: VEP keep failing + enhancement suggestions

Veeam Logoby Dima P. » Mon Oct 17, 2016 5:37 pm

it still backed up everything in the whole volume

Looks like VSS exclude operation has failed. Hold on, I'll check with the QA guys. Will follow up shortly.
Dima P.
Veeam Software
 
Posts: 6698
Liked: 479 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: VEP keep failing + enhancement suggestions

Veeam Logoby bhwong » Tue Nov 08, 2016 1:56 am

Hi Dima,

I'm still trying to get it to works in my environment and I realize 2 issues:

1. I tried to make it run as volume level backup and I did saw it indicating volume level backup under the Files screen. However, when I reached the Summary screen, it show Backup mode: file level backup. (is it possible to attach my screen capture here?)

2. I have reconfigured my 2TB selected folders backup to 192.168.100.245, but the log shows that the total backup size is 6.46TB and it's still trying to backup to 192.168.100.241 which was my original configuration. No wonder it fail everyday.
bhwong
Enthusiast
 
Posts: 92
Liked: 2 times
Joined: Thu May 24, 2012 9:57 am
Full Name: Boon Hong Wong

Re: VEP keep failing + enhancement suggestions

Veeam Logoby Dima P. » Tue Nov 08, 2016 10:51 am

1. I tried to make it run as volume level backup and I did saw it indicating volume level backup under the Files screen. However, when I reached the Summary screen, it show Backup mode: file level backup. (is it possible to attach my screen capture here?)

Just share the link to the screenshot.

I have reconfigured my 2TB selected folders backup to 192.168.100.245, but the log shows that the total backup size is 6.46TB and it's still trying to backup to 192.168.100.241 which was my original configuration. No wonder it fail everyday.

So, you mean the destination path was not updated in the job settings?
Dima P.
Veeam Software
 
Posts: 6698
Liked: 479 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: VEP keep failing + enhancement suggestions

Veeam Logoby bhwong » Wed Nov 09, 2016 4:31 am

Here's the screen capture that show volume level backup:
Image

At the end, it show file level backup instead:
Image

The new saved destination path to: 192.168.100.245\backup
Image

The actual path still return to previous destination: 192.168.100.241\veeam:
Image
bhwong
Enthusiast
 
Posts: 92
Liked: 2 times
Joined: Thu May 24, 2012 9:57 am
Full Name: Boon Hong Wong

Next

Return to Veeam Agent for Windows



Who is online

Users browsing this forum: Bing [Bot] and 1 guest