Veeam Newbie... a Few Questions

[nick3young]

Hi All,

I have been trialling 'Veeam Backup and Replication 7 for VMWare' for a week now and have some questions. I would be most grateful for answers to as little or as many of these as possible......

1. We backup to a QNAP NAS box (disk to disk). Why, when we specifiy a 'Shared Folder' for the Veeam backup repository does it state that it doesn't 'support storage agent'. What does that mean?

2. Is it better to have ONE backup job containing all of your virtual servers or a job for each of your servers. What is best practice?

3. I have been carring out this trial using the 'reverse incremental' system. Our first full backup went through 100% perfectly. I was impressed! However, on subsequent backups (the incrementals), one of our servers has been failing. The error we get is "Client Error: End of file Exception from the server: The process cannot access the file because it is being used by another process. Failed to create or open file." This is followed by the path to the backup folder on the NAS box. It then attempts to backup the server again... which goes through fine on the 2nd run.

4. Our incremental backups are taking nearly as long as the initial full. The processing rate for the full was 47 MB/s (bottleneck = source). The processing rate for the incrementals is 5 MB/s (bottleneck = target). Why is the processing rate so much slower and how can I improve it? Does this link to my first question and this support 'storage agent' on the 'Shared Folders'?

Thanks in advance for answers to any of those!
Nick

[foggy]

Hello, Nick. Welcome to the Veeam Community Forums!

1. We backup to a QNAP NAS box (disk to disk). Why, when we specifiy a 'Shared Folder' for the Veeam backup repository does it state that it doesn't 'support storage agent'. What does that mean?

There're three types of backup repositories within Veeam B&R: Windows or Linux-type repositories are agent-enabled, i.e. Veeam agent is installed on the repository server and performs data processing activity on the repository server itself. In case of CIFS repository, the agent cannot be installed so it is recommended to have a proxying server so that the agent could be installed there for optimal data processing performance. If you decide to use CIFS shares as a backup target without proxying server, then it will increase your data traffic between proxy and the NAS box, moreover the rebuild traffic (synthetic full) will not be kept locally on repository and will be pushed back and forth over the network decreasing the overall job performance. Additionally, throttling rules (if you decide to use them) will not be applied as they require our run-time on the NAS box for communication. Finally, with all network glitches that might happen during the backup job run network shares cannot be considered as the best possible backup target.

2. Is it better to have ONE backup job containing all of your virtual servers or a job for each of your servers. What is best practice?

Please review this thread for recommendation on grouping VMs in the jobs.

3. I have been carring out this trial using the 'reverse incremental' system. Our first full backup went through 100% perfectly. I was impressed! However, on subsequent backups (the incrementals), one of our servers has been failing. The error we get is "Client Error: End of file Exception from the server: The process cannot access the file because it is being used by another process. Failed to create or open file." This is followed by the path to the backup folder on the NAS box. It then attempts to backup the server again... which goes through fine on the 2nd run.

Do you have Veeam B&R Patch 1 installed? This kind of issue was addressed by this patch, so probably installing patch will help.

4. Our incremental backups are taking nearly as long as the initial full. The processing rate for the full was 47 MB/s (bottleneck = source). The processing rate for the incrementals is 5 MB/s (bottleneck = target). Why is the processing rate so much slower and how can I improve it? Does this link to my first question and this support 'storage agent' on the 'Shared Folders'?

Yes, this is actually expected for CIFS target on a NAS box, see the answer to the first question. Moreover, apart from the fact that synthetic takes place over the network, NAS is typically not considered a good storage for synthetic activity as it puts much more heavier load on the storage. Add the proxying server to your CIFS share, specify it as backup repository, this will enable synthetic backup activity to be performed locally on the backup target instead of sending data across the network. Further step in case if this is not enough, will be to switch to forward incremental mode.

Thanks!

[Vitaliy S.]

Adding my 2 cents.

2. Is it better to have ONE backup job containing all of your virtual servers or a job for each of your servers. What is best practice?

I would say that with parallel virtual disk and VM processing it is worth adding VMs into a single job, however you should also keep in mind the size of the backup file produced. Generally 8 TB full backup size should be fine to manage and archive (if needed).

[nick3young]

Thanks for the speedy replies.

I am currently backing up 10 VMs with a total full size of under 3TB; so I'm assuming I'm ok to have them all in one job at the moment. I will read through the recommended thread in more detail too.

Thanks for the very detailed response Foggy. I understand what you are saying about the disadvantages of backing up over a network but I was under the impression that backing up to disk over a network (e.g. NAS) was becoming the 'norm' these days? You have got me thinking though....! What is the best option? Backing up the server itself and then to tape afterwards? Surely any other option would be over the LAN?

The reason we currently back up to the QNAP over the LAN is because it is in an entirely different building. Would you recommend backup up locally to the Veeam server first and then setting up some sort of script to copy the VBK & VRB across?

[foggy]

I'm not saying that backing up to the repository located over network is not normal. I'm just saying that having backup repository on a CIFS share, where you cannot install Veeam agent on, you are increasing the amount of data to be sent. If you have Veeam agent on both ends (proxy and repository), the data is transferred between them in a highly compressed format, and this is not the case with CIFS-type backup repository. Another thing is synthetic backup operation, which is not performed locally on the repository, if there is no Veeam agent on it. That is why it is recommended to set up a proxying Windows server with mounted NAS to it and add it to Veeam B&R as a Windows-type repository. This will make your repository agent-enabled with all subsequent benefits.

[nick3young]

By 'mounting the NAS' do you mean, mapping it as a drive? And then choosing that as the backup location? If so, could I not just map the drive to our existing Veeam server to solve the problem?

Hmmmm.... I'm now considering one of these two options then to move forward. Sorry for the continuous questions but I want to try and make sure I get this right......

1. Have a physical dediciated box with Veeam installed on which also backs-up to itself. Does this cause a bottleneck issue though having the same box as both the proxy and repository? I would then transfer the backup across to the offsite NAS once the backup has completed.
2. Install a server next to our NAS box with the Veeam agent on. I am assuming here pysical location doesn't really make much difference?

[veremin]

Would you recommend backup up locally to the Veeam server first and then setting up some sort of script to copy the VBK & VRB across?

Actually, if you stick to this scenario, you won’t even need to use script a for it, since with the version 7 there is a new type of jobs called “backup copy jobs” that will handle the process of copying backup data to secondary repository for you.

Thanks.

[foggy]

By 'mounting the NAS' do you mean, mapping it as a drive? And then choosing that as the backup location? If so, could I not just map the drive to our existing Veeam server to solve the problem?

Sure, you can present it through iSCSI to the Veeam backup server. Another option is to add the NAS as a Linux-type backup repository (in case it has ordinary Linux installed as SSH and Perl are required).

1. Have a physical dediciated box with Veeam installed on which also backs-up to itself. Does this cause a bottleneck issue though having the same box as both the proxy and repository? I would then transfer the backup across to the offsite NAS once the backup has completed.

With the backup load you're going to assign to it (10 VMs), I don't think there will be any problems running all-in-one. Btw, what kind of link do you have between buildings?

[nick3young]

Brilliant. Right, so this is what I think I am going to do:

It just so happens that we have a NAS box with Windows Storage Server installed on it. I'm going to stick Veeam on that and back everything up, using one job and reverse incremental to it's own storage array. Then I will use 'backup copy jobs' (thanks for the tip v.Eremin) to send our backups across to our off-site QNAP.

Foggy, we use a single 1gbps fibre link between buildings. We have considered terminating some more cores to have a dedicated link for backups/restores.

[foggy]

Keep us updated on how it is going.

[nick3young]

Will do. We're all setup and ready to go tonight....

[nick3young]

Last night's full was 2 hours quicker than backing up over the network. The real interesting bit will be how much quicker the incremental is tonight.....

I've set up the 'Backup Copy Job'.... I just have a quick question about it though. The job runs 'continuously' so I imagine it is constantly checking the backup folder for any new files to copy across. If you don't want it checking all day, a schedule can be set, so e.g. only check backup folder between 1am and 6am. What I don't understand is, on the 1st screen of the wizard you have to select how often backup copies are created. So for example, 1 every day at 4am OR 1 every minute(?!) at 12pm. What is that bit all about? If it is contantly checking during the scheduled times and copying the data across whenever there is new.... what is the point in that setting? Thanks again.

[veremin]

This setting refers to the synchronization interval. The synchronization interval is a period during which the backup copy job is supposed to copy a VM restore point from the source to the target location. For instance, if the synchronization is set to 1 the backup copy job will create a new synchronization interval once a day. And the time you set defines the start of the synchronization interval.

For more information regarding this option, please, refer to this section of our Help Center.

Thanks.

[nick3young]

Thanks for the reply. I had read that page actually but it didn't really shed any light on it for me! Sorry if I am sounding stupid!

I imagine the vast majority of people only perform one backup in a 24hr period? And if the 'Backup Copy Job' is continuously scanning for changes... what would be the point in having more than one synchronisation period in a day? What is the point in having it check for changes to the backup every four hours, for example?

[veremin]

In case of daily backup jobs, there will be, indeed, no point in setting synchronization interval to 4 hours. However, if user runs backup job every 4 hours and want to copy resulting restore points, the 4-hour synchronization interval will be exactly what answers user’s requirements.

Thanks.

[nick3young]

Ah ok fair enough. I'll keep mine at one day then. Thanks for your help.

[foggy]

Another thing that should be mentioned, is that the backup copy does not copy files but rather rather synthetically creates restore points in remote location from the changed blocks extracted from the source storage.

[nick3young]

Ah right, that's interesting Foggy. So does this mean I may encounter peformance issues on the QNAP again then if the restore points are synthetically being created on it?

I have now ran a full backup and a reverse incremental using the local Windows Server as the repository and a backup copy to the QNAP NAS. Here are my findings:

- The full backup was 2hrs quicker when performed to local storage (down from 15hrs to 13hrs).
- The reverse incremental backup was, as expected, MUCH faster (down from 14hrs to 3.5hrs)
- The backup copy job has now run twice and it takes a LONG time. It was getting on for 13hrs before our standard backup interrupted it. At the time I thought it was a simple 'copy file' job but now I know about this synthectic processing, I assume that is causing the performance issue.

What I don't understand is: the files in the 'backup copy' folder dont match the size in the actual backup folder. Surely they should shouldn't they if they have been correctly syncronised (which they have)? All I want is a simple 'copy files across' job. Perhaps I should stick to just using a standard batch file...? Having said that though, I do like the idea of it synthetically creating restore points if it removes the need to copy the full file across every night... but not if it is going to cause performance issues on the NAS box. Thanks.

[foggy]

Ah right, that's interesting Foggy. So does this mean I may encounter peformance issues on the QNAP again then if the restore points are synthetically being created on it?

Yes, backup copy job performance issues are explained by this fact, indeed. However, this typically is not considered as critical as running regular backup jobs to QNAP as backup copy jobs are performed outside your backup window. Was it able to transfer the initial full? Or are you saying it is not able to sync even increments within the allowed time?

What I don't understand is: the files in the 'backup copy' folder dont match the size in the actual backup folder. Surely they should shouldn't they if they have been correctly syncronised (which they have)?

Regarding backup sizes, please see this thread for explanation. Your regular backup job uses reversed incremental mode, while backup copy job is always forward incremental.

All I want is a simple 'copy files across' job. Perhaps I should stick to just using a standard batch file...?

Probably, simple file copy using some 3rd party utility like rsync would be justified in case of such target.

[yizhar]

Hi.

* I'm against single job for 3tb.
You'll be better with 3-4 jobs.
Put large file and mail servers as single VM per job.
Other servers combined depending on your perferences.
Using several separated jobs gives you more flexibility, and better management of VBK files.
It is also safer - a corrupt or busy VBK file won't affect other jobs.

Regarding NAS -
I recommend backuping up to local disks on the backup server itself.
From there you can use a "copy job" to save additional backup to NAS.

Yizhar

[nick3young]

Thanks for your tips Yizhar. I think I will stick with one job for now just to keep it simple. I have started backing up to the local disk followed by a 'copy job' as you say though and I agree, this way is much better.

As for the 'copy job' though... I think I am just going to setup my own 3rd party job as Foggy mentions above (e.g. rsync). My backups are going through fine, but my backup copy jobs keep failing. I suspect this is down to load on the server and it not being able to cope. Its a shame that Veeam don't have a 'simple copy' job to back your 'backups' up off site. It gets a little over complicated in my opinion with all this converting 'reverse backups' to 'forward backups' for 'backup copy' jobs... I can't understand why this is the case really.

If I use something like rsnc to simply copy our backups across to another site say once a week... will this be OK? If, worse case scenario, we have a fire and the servers burn, will I be able to install a brand new host, install VMWare (etc), install Veeam, copy the backups across from remote site... and then recover? I take it I would need a 2nd backup of the 'VeeamConfigBackup' too?

[Vitaliy S.]

If, worse case scenario, we have a fire and the servers burn, will I be able to install a brand new host, install VMWare (etc), install Veeam, copy the backups across from remote site... and then recover?

Yes.

I take it I would need a 2nd backup of the 'VeeamConfigBackup' too?

No, need to backup this file, just import your backups to the new backup console to start the restore procedure.

[yizhar]

If I use something like rsnc to simply copy our backups across to another site say once a week... will this be OK?

Hi.

Using RSYNC or other 3rd party tool might cause data corruption, because:
You have no protection against an error during sync - what happens if the job copies only half of the changes and then something happens?
Do you have a solid monitoring and verification of the backup copy?
This is also related to one a problem with large single VBK files vs several smaller ones.
Using several VBK files you can reduce the risk for corruption, and also might have better results with the copy jobs.

Using RSYNC will probably work in most cases but not all, and I think this is not good enough for backup whether onsite or offsite.

I have no experience with the new "copy job" in Veeam V7, but if you have troubles with it I suggest that you contact Veeam support and open a case, in addition to asking in this forum.
Some configuration changes and tweaks will probably solve the problems.

Yizhar

[Doug.Butler]

Sorry to boost this older thread, but I'm just getting started with Backup Copy job after recent upgrade to V7. My target is EMC VNXe3100, using CIFS share, which I now understand (after reading this post and several others) is probably my main problem with the performance of the Backup Copy job. (More than 24 hours to move about 1.5TB of backup data, which meant that some VM's got skipped on the first pass due to it not happening within the 24-hour backup cycle.)

So my question relates to switching from CIFS to Windows-type repository, but using the same storage on the VNXe and NOT using iSCSI. You've mentioned using a Windows Proxy. Obviously can't install Windows on the VNXe! And I do not want to mess with iSCSI. The original poster (nick3young) asked similar question... can I just map a drive on my Veeam server, and then create another repository on that server pointing to that mapped drive? This is the main Veeam server that has the console, the database, and is acting as proxy and repository. Can it have a second repository pointing to a mapped drive using CIFS? And will this do any good for improving the performance of a Backup Copy job using this repository? Or should I just always expect slow performance if I'm using CIFS to a non-Windows NAS?

[foggy]

Can it have a second repository pointing to a mapped drive using CIFS? And will this do any good for improving the performance of a Backup Copy job using this repository?

Sure, you can do this, however the amount of data to be sent over the network (and, hence, performance) will be the same. What kind of link do you have between Veeam backup server and repository? What backup copy job operation mode do you use (direct or WAN accelerated)? Were you considering capability to seed the backup copy job?

[Doug.Butler]

Sure, you can do this, however the amount of data to be sent over the network (and, hence, performance) will be the same. What kind of link do you have between Veeam backup server and repository? What backup copy job operation mode do you use (direct or WAN accelerated)? Were you considering capability to seed the backup copy job?

I have gigabit connectivity at both the Veeam server and the VNXe. I'm using "Direct" mode - the Veeam server and the VNXe are on the same subnet, same LAN. I could 'seed' the backup copy job, I suppose, if I needed to. I just let it run overnight (fresh instance - I deleted the first set of backup copies and the job, and rebuilt the backup copy job).

I'll have to evaluate whether the Backup Copy job's duration, for the typical daily incremental backups, will be OK. I am sure it will be, but would like to avoid having it take 24+ hours if I ever need to delete the copies and re-start the whole cycle. Maybe that's where the "seed" will come in, but I can't see a way to get data onto the NAS share via any kind of removable media, so it will probably just be something I have to live with, if I want to utilize the VNXe's CIFS share as the repository storage.