Best Practice for putting Veeam Backups on Tape?

[flavor4real]

Hello guys,
A Issue cause me to lose all of our veeam backups which were on two drives on the ESX. Now, I have to start from scratch. While Veeam backups were running and backing up to these two drive, we ran netbackup to copy these two drives to tape. I have a few question:

1. Could the Netbackup job during the veeam backup could cased a corruption on these ESX drives?
2. What are is the best practice to backup these two drives of the ESX?
3. Is there a way to setup a data replication to dataDomain in Veeam?

thx

[Gostev]

Hello,
1. No, as long as NetBackup is read-only and does not try to modify the file content.
2. Actually, the best practice is to never backup to VMFS file system (as you are doing, if I am reading correctly), because of how hard it will be to get your backup files out of there if something happens.
3. Performance wise, best is to simply copy backup files from disk to DataDomain via post-job script.
Thanks!

[flavor4real]

Thanks for your respond Gostev.
In concern of the post-job script, can you point mwe to a direction where I can get some mor enformation about it. Thanks

[Gostev]

Writing to Tape Help

[flavor4real]

I might have a kinda unlogical question now. The Veeam Backup & Replication Application is installed on a Virtual Machine, and the two drives presented are virtual too. The VBK's and VIB's are getting created on these two drives. Is that a good practice? or should Veeam B/R be installed on a physical server, also backing up to a Physical server?

[Gostev]

This effectively means you are storing your backups on VMFS, which as I explained is not a good practice. But physical server is not required to be able to use better backup targets. With Veeam Backup running in VM, you can use NTFS formatted iSCSI LUN, or NAS box, or even regular share on some physical computer - all these would be safer backup targets. Or if you have NFS datastore on ESX, move those Veeam VM VMDK files there. Use anything but VMFS.

[flavor4real]

I understand and that's what I assumed. Thanks for the Info and help. I appreciate it..

[ec-inflxx]

This effectively means you are storing your backups on VMFS, which as I explained is not a good practice. But physical server is not required to be able to use better backup targets. With Veeam Backup running in VM, you can use NTFS formatted iSCSI LUN, or NAS box, or even regular share on some physical computer - all these would be safer backup targets. Or if you have NFS datastore on ESX, move those Veeam VM VMDK files there. Use anything but VMFS.

My Veeam Server is a VM..... So if I have a DAS (Dell MD1120) that is directly attached to the ESX host and presented as to VMFS volumes and I am using those as my targets for the Veeam backup, that is not considered good practice? I figured the performance would be much better doing this then attaching the disks to a physical server and sharing it out via iSCSI or regular network share? Or will the actual performace be the same for the backups?

[Gostev]

There is certain risk. If your VMFS LUN corrupts, you will be on your own to get your backups out of there. If your NTFS LUN corrupts, you will have a choice of tools and companies providing data recovery services.

But as long as you additionally copy and store your backups offsite (which you really should be doing anyway, no matter what is your primary backup target) - then you can just stick with how you do this today, since you do have plan B.

Frankly, I think performance will not be so much different on 1Gb LAN.

[ec-inflxx]

Ok, thanks for the input. We are backing up offsite to tape. I was toying with the iSCSI route because it would be nice to be able to use VMware's vMotion or HA if needed. But I was worried about performance of the backups and I think was told connecting it directly to the host was the best way.

[flavor4real]

Good Morning,

We use Veeam v5 to backup all 66 VM's to Disk. Now, we are thinking to implement backup to taper for offsite storage. We do have a DataDomain where when we could run a PowerShell script on the Veeam to backup to it. Now, what would be the best practice to move the data off to tape?

We could have Netbackup directly backing up all 66 VM's and since Netbackup had direct control over the Tape Library.

According to a college of mine, "The only way I can think of is have a script copy the Veeam backup files to a location that is backed up by NetBackup. In Linux, we used a program called 'rsync' to copy only changed files to a different location. If there is a rsync version for Windows, it might work for us."

How do you suggest or do your backup to Tape?

thanks

[Gostev]

Hi, I've moved your previous post into the new topic, because it was out of topic in the original thread you decided to place this to (comparing Veeam with NetBackup). Besides, you have created this very topic yourself, and it talks about the exact same thing... in future, please try to avoid creating duplicate topics, or derailing existing topics on different subjects.

As for your main question, I gave you a good link above that explains some alternatives. There are usually a few ways to choose from depending on capabilities of specific tape backup program (I am not so familiar with NetBackup, but there are plenty of examples posted for BackupExec for example that you can find with search).

One thing for sure, you definitely do not need to copy backup files around. Just point NetBackup to Veeam backup folder, I am sure it lets you choose which files to backup. And if you really, really need those backup files to be in a separate folder (not sure why would you), then again - do not copy, instead just create a soft link to file with PowerShell script run as post-job activity.

I think it will be beneficial if you update everyone with what approach you have finally chosen to go with for NetBackup 7 and why, because most of the current topics on tape backups seem to be dedicated to BackupExec.

Thanks!

[gvinpin]

We have around 500 VM's in 2 Data Centres, and two DD860 DataDomain Appliances.
Veeam folders are replicated between appliances and we taking tape backups from second appliance to a tape library using EMC Networker software.
We keep a month worth of backups , using Incremental Backup and Active Full backup on a monthly base. Deduplication done on DD860, as it has a global deduplication pool. No compression on Veeam as well.
Networker is configured to take only *.vbk files, so we will not backup any incrementals.
We are getting deduplication ratio of seven times.
All going well, but our monthly backup to tape is huge, and takes around 50 tapes. (LTO3)

If anyone has any recommendations on how to improve this scenario, I would really appreciate.

[Gert1963]

I had myself to start with VEEAM in combination with Quest Netault this week.

This solution is a way to start the backup, it wil let VEEAM create a inclusionlist for NetVault.
Unfortunally NetVault requires the inclusionlist to be in Unicode format (starting with FF FE as first 2 characters).
Microsoft CMD can create unicode output, but these do not contain the filer-header.
I created a 6 byte file (although 2 byte should be enough) creating the header (2 bytes) and a CRLF (4 bytes)
(I use Write to create a unicode file and with ultraedit I stipped most data exept for the header and the CRLF as required minimum for UltraEdit...).

In the postbackup, I let VEEAM call a script (E:\createlist.cmd) which contain these commands:

echo off 2>error.txt
setlocal
copy /y e:\unicodeheader.ro e:\inclusionlist
cmd /u /c dir /b /s e:*.vbk >>e:\inclusionlist 2>e:\error.txt
exit 0

Within NetVault I create a client for VEEAM to use only the include list as backup-selection and let it run only one day after a syntatic full. This post-process however could run every backup... but at least as postproces of the syntatic full backup.
An other client resource can handle the SQL database and regular filesystems.

Obvious if I got powershell working (smarter selection creteria), I still have to convert it to unicode format for NetVault, so only "dir /b /s e:*.vbk" could be replaced.

For all you people.

Gert de Vente / Telindus - ISIT

[slidek9]

We use EMC networker to backup the .VBK once a week; and use this script to create a legato save set list:

dir "V:\Backups\Group 1 - Windows 2003\*.vbk" /OD /B /S > "V:\Scripts\Group1\Group1.txt"

for /f "usebackq delims=;" %%a in ("V:\Scripts\Group1\Group1.txt") do (
echo %%a
exit /B
) > "V:\Scripts\Group1\Saveset-list-group1.txt"

Also we run a job to index all the file servers once a week to avoid having to pay for enterprisw (the date format is for UK not usa, this is important);

This is stored on the C:\scripts dir of the file servers:

set fname=%computername%#%date:~-4,4%-%date:~-7,2%-%date:~-10,2%#%time::=-%

rem Please place paths you would like indexed below
rem Please notice the 1st list contains a single >
rem while the rest of the path lines contain double >>

echo Silent output now while crawling please be patient.

echo %date% %time% > C:\scripts\%fname%.log
dir N:\Shares\ /ON /TW /S >> C:\scripts\%fname%.log
dir K:\Shares\ /ON /TW /S >> C:\scripts\%fname%.log
dir L:\Shares\ /ON /TW /S >> C:\scripts\%fname%.log
dir E:\Shares\ /ON /TW /S >> C:\scripts\%fname%.log
dir P:\Shares\ /ON /TW /S >> C:\scripts\%fname%.log
dir X:\Shares\ /ON /TW /S >> C:\scripts\%fname%.log
echo %date% %time% >> C:\scripts\%fname%.log

And these types of scripts are called from a 'master' script' on the backup server:


set cname=<servername goes here>

psexec \\%cname% c:\scripts\crawl.cmd

rmtshare \\%cname%\copy$ /delete
rmtshare \\%cname%\copy$=c:\scripts /users:25
robocopy \\%cname%\copy$ C:\things\test *.log /MOV /R:1
rmtshare \\%cname%\copy$ /delete

<more servers here...>

set idate=%date:~-4,4%-%date:~-7,2%-%date:~-10,2%

pkzipc -add -max C:\scripts\%idate%.zip C:\scripts\*.log
del C:\scripts\*.log /q

Uses 7zip commandline and DOS

Generates a zip with the dir listing for the week, then we use a freeware tool to search though the zip when we are looking to see which tape set the file is on.

[AlexL]

You can also create a unicode file in notepad, just use save-as.

Back to the original question: can tape backup cause disruption while VEEAM is running?

My answer: YES

From experience, we backup around 200 VM's to disk daily and do a weekly .vbk backup from disk to tape. The amount of I/O that a simultanious VEEAM backup (mostly writes) + tape backup (reads) generate might cause to disks to be overloaded, too much IOPS, which might cause delays and/or timeouts which might cause either veeam or your backup product to generate failures.

We've had this happen many many times, did a lot of research, also worked with support but in the end it turned out to be a simple concurrency issue, not software but a hardware problem, or rather a shortage of spindles.

[flavor4real]

[merged]

Hello,
In my scenario, I have a Veeam Backup VM which backups up to a total of 4 Mapped Raw LUN's, each 2TB. We would like to push these backups onto tapes. Out of previous experience, we are not going to pull the original vib's, vbr's. We did that before and had Netbackup copying the backups file while Veeam processed backups jobs and it crashed, and we lost all vbr's/ vib's.

We are thinking of duplicating the vbr's and vib's to a different storage location and then have net-backup pull the data on scheduled times.

the Replication part of Veeam does allow to replicate VM's itself but can I replicate a RDM copy to a different location?

What other ways do I have to get this accomplished?
Thx

[flavor4real]

Hello Alex,
I've lost a entire stack of backups when we tried to have Net backup to copy veeam backup files onto tape. I'm not thinking of replicating the Mapped Raw LUN's to a different location and have Net backup pull from that location.

Do you have any suggestion how I can accomplish that? I thought about the Replics option under veeam but it only does complete VM replicas. Therefore a post script to copy file would be great, but I already have several Veeam backups jobs daisy chained over a post-script action, to run in sequence. so that's not an option. Robocopy would be an option but, I have four 2TB drives and this will take some time.

Any ideas?

[AlexL]

Simple solutions are usually better, if you can backup VEEAM to disk at night and from disk to tape during the day so there is no overlap there should not be any issues. Of course I do not know what exactly happened in your specific case but it's fairly certain that generating a high read load (=tape backup) while doing a high write load (=veeam backup) might very well cause issues as it did with us on numerous occasions. We've added disks (spindles for iops, not for space) and made more sure that there is no overlap, no problems since and very good backup speeds with both veeam and tape backup since they don't bother each other.

[flavor4real]

hmm,... that would not work because we have jobs starting from 6am to 6pm, at times every 3 hrs and others every 6 hrs. The last job usually finishes at 0900PM. We would not have enough time to run anything during night time.
is there a way to actually replicate? I mean that a backup job backup the implemented VM's to two different storage locations?

[AlexL]

err, and you also have veeam jobs at night? if not then you could do the tape backup during the night, perhaps a bit harsh but it sounds to me that you have a design that does not fit your current infrastructure, change either one instead of trying to ll things together with duct tape.

but root cause analysis first, what is causing the disruption, is it disk overload or is it some other cause.

[flavor4real]

yeah, ... looking for setting up net backup to backup all on tape only on weekends. wish there would be a function in veeam to backup to two different location at the same time.

[foggy]

You cannot backup to two different locations with a single job. You might consider rearranging your backup schedule to perform two jobs with different targets though or use some kind of disks rotation to be able to safely copy to tape from one disk while the other is used for current backups. Thanks.

[rahlj]

[merged]

I know this has been a topic for discussion but i dont see an "answer" per say. We have a Networker server and I want to archive my Veeam backups to tape and clear disk space on my server. I dont necessarily need the Networker specifics I just need how is this supposed to be accomplished? File backup using Networker? How does Veeam know that the files were moved to tape? Thanks - Jason

[Vitaliy S.]

Hello Jason,

You should correlate your backup strategy with tape offload scheduler, lots of our existing customers are using archive bits to mark backup files which were archived to tapes. If you want to clear backup destination storage, please use retention policy settings for that.

There is some information on Networker specifics on the previous page, please have a look.

Btw, if you want to search through your archived backups, here is an existing discussion covering all the details: Indexing servers and tape

Hope this helps!

[zak2011]

[merged]

Since Veeam does not have tape integration and i will be shifting my backups from Backup Exec 2012 to VBR6.1 soon, my organization prefers to have the option of backing up the VMs to tape. What would be the best way to be able to get the BE to backup the .vib files to tape?
Thanks