Backing up IBM Lotus Domino 9

[massimiliano.rizzi]

Hello experts,

we've had a Veeam Backup & Replication prospect asking us how Veeam can consistently handle IBM Lotus Domino 9 backups.

In this video https://youtu.be/dRGRi_fYK4Q Veeam’s Solutions Architect Andreas Neufert was using IBM Lotus Domino 8.5.3 in his lab.

Does the information provided still apply to IBM Lotus Domino 9 as well ?

Thank you and have a great day.

Massimiliano

[Dima P.]

Hi Massimiliano,

I assume, nothing changed and this youtube guide is still valid. To be 100% sure I’ve forwarded your question to Andreas, so let’s await for the answer. Thanks.

[Andreas Neufert]

Correct, nothing needs to be changed.

As IBM still do not have any command to bring the whole domino server to a consistent state you have 3 options for an image level backup:
1) Crash consistent backups. Not really a good idea but domino self healing databases are pretty reliable. Back in the old times it was the only option to do online backups.
2) Shutdown the Domino Service. You can do it for example with one of the cluster members. If you don´t use domino cluster, this will interrupt the service.

Code: Select all

net stop "Lotus Domino Server (DominoData)"

Code: Select all

net start "Lotus Domino Server (DominoData)"

3) Close all the connection and write the RAM to the disk. This is somehow pseudo consistent as a client can reopen a database, but it is way better than crash consistent backups.

Code: Select all

c:\Domino\nserver.exe -c "drop all"
timeout /t 10 /nobreak
c:\domino\nserver.exe -c "dbcache flush"
timeout /t 30 /nobreak

Please check domino console output if it works as expected or if you have to change the timeouts somehow.

As Veeam can only work on a image level, these are the options. For a Domino Admin the question is, why someone should do this as Veeam do not use the official API.
1) Fast Restore of the whole Server. Instant VM Recovery speaks for itself.
2) Your organisation decided to do Veeam Image Level backups anyway, so it is in any way a good idea to bring this restore point to a consistent or at least pseudo consistent state.
3) Veeam Backups are pretty fast and you can do these more frequently as traditional domino backups.
4) Veeam SnapShot orchstration with even better RTO and RPO
5) Fast restore (copy back) of individual mails
Install a Notes Client with a ID that can access the production mail databases.
Start Instant File Level Recovery and double click the NSF files under the c:\VeeamFLR path and subfolders (all domino server disks are mounted there.
Domino opens and you can copy a mail (index based search should work as well).
Open the original mailbox and paste the mail wherever you like.
This process takes overall 2-3 min as the mailbox is not copied/restored compleatly.

Ad Domino Admin you can as well use Instant File Level Recovery to copy or access the databases you like.
5) With SureBackup a automatic boot, IP and Domino Service restore verification can be activated, so you can check if you really be able to
bring back the service online in case of an emergency. Yes, backup and restore process is tested, but as well the OS and Domino itself if they will boot after a restart/restore.
6) On Demand Labs can be used to bring one or many domino servers online in a sandbox environment and you can use it as a test environment for patch updates or development stuff.
It uses the last restore points in most cases, so you can test with production data.

Veeam has no option to work with Domino Logfiles. It is a good idea to use existing Domino Backup tools for everything that needs logfile handling at backup and restore.

[massimiliano.rizzi]

Hello Dmitry/Andreas and good day,

@Dmitry thank you very much for taking the time to reply to my question and forwarding it to Andreas

@Andreas thank you very much for the quick and very detailed response

Have a great rest of the day !

Massimiliano

[netamego]

Hi everybody!!!

We are a company with CommVault but at this moment we are testing Veeam as alternative backup solution. One of Veeam's advantage is its client-less approach. We are testing backup of VMware Virtual Machines with Domino Servers and all seems very confusing for me and i couldn't find specific documentation for this type of backup.

I saw Andrea's Webminar (https://www.veeam.com/videos/backing-up ... -4867.html). As i understand, we could do Online Lotus Domino Backup with pre-script based in two commands:

nserver.exe" -c "drop all"
nserver.exe" -c "dbcache flush"

Then, Vmware Snapshot runs with Lotus Domino Databases in consistent state. This approach is right in many scenarios but our Lotus Domino Databases are highly accessed all the day so is possible that while Vmware Snapshot is in place new sessions arrive or sessions are re-established in Lotus Domino Server and the backup save Domino databases in not consistent state.

So, are there any task we can do to guarantee Lotus Domino Databases consistency during backup without stop Lotus Domino Services?. Are there any specific documentation for backup Domino Servers highly accessed?

Thanks a lot.

[TobiKI]

Hello Guys,
i too have seen the best practise shown by Veeam Expert Andreas Neufert.
I am currently in the evaluation process of the Veeam Backup & Replication 9.5 Suite.
My Company is planning on globaly changing their backup solution to Veeam Backup & Replication 9.5.
Well unfortunatly we also have IBM Domino 9 Servers.
I followed the guide provided by Veeam Expert Andreas Neufert.
The backupjob is configured with application aware processing and a domain-administrator account wich is also an administrator on the Domino Server.
The script is located on my Veeam Backup Server. I can see from the logs that the script is beeing copieed to the admin share of my Domino Server, but the backup job is running into a timeout waiting for the script after 10 minutes.
The script is working under the exact credentials when executed on the server itself. For troubleshooting purposes i added a message to my script before and after the script aswell as an echo %Username% wich logs into a txt file. I can see the script is beeing executed on the server because the message box pops up! It is beeing executed by the local computer account of the Domino Server. After not getting any further with the Batch files i switched to powershell.
I created a powershell script wich sends the "Drop all" and "dbcache flush" command. This script was testet localy and works! This script is called and startet by the pre freeze script with the Domain Administrator Credentials.
Again, this is working perfectly when executed on the Domino Server localy but not as pre-freeze script when used by Veeam. With this setup the backup job no longer times out because of the script, but on the Domino Server the commands are not beeing sent.
Also: UAC is deactivated on the Server!
I am running out of Ideas.
The Veeam Service Account Settings are specifyed as LOCAL SYSTEM account (recommended, used by default).

I would greatly apriciate your advice.

Kind Regards Tobias

[Andreas Neufert]

Please use the same account for Veeam InGuest processing as you had used for the local script test.

Did I got you right, that the script runs perfect within Veeam Job when you only use the echo to txt file lines?

[TobiKI]

Hello Mr. Neufert,

thank you for your reply. I have specified the same account i used for the local tests in the Guest Processing of my Backup Job.
To your second question: No, it does not work perfectly fine.
For clarification,script execution seems to run perfectly if i let the script put out a Message Box for example.
I used the echo to txt lines in my script to check wich account is used to execute the script.
In my script i used Message Boexes to determin at wich point the script fails.

Msg * "TEST THIS1 "
"D:\Program Files\Lotus\Domino\nserver.exe" -c "drop all"
Msg * "TEST THIS2 "
timeout /t 5 /nobreak
Msg * "TEST THIS 3"
"D:\Program Files\Lotus\Domino\nserver.exe" -c "dbcache flush"
Msg * "TEST THIS 4"
timeout /t 5 /nobreak
exit 0

Only the Test 1 message box appears.
Well after trying several scripts i created a scheduled task with the SAME credentials as the one used inGuest Proccessing to execute the script.
Veeam triggers this with the pre freeze bat wich contains :

schtasks.exe /run /tn "LNDBFLUSH"
exit 0

This is working fine but ideally it would work as you described.

Thank you again for your support.
Regards

Tobias

[Andreas Neufert]

Strange, as the script itself is working and at Veeam Job the first line is executed. Maybe some Group Policy restrictions?

[TobiKI]

Yes, very strange.
I will look into the Group Policies later.
Thanks again for the tip.
Regards

Tobias

[Andreas Neufert]

Oh... or did you used an editor in Unix mode to edit the script... then the new line command is different and only first line is executed.

[TobiKI]

No, Script was written in Powershell ISE.
I am stuck with using the Scheduled task option.
I kind of gave up now.
I will report back if i find another solution.
Regards

[zuldan]

We are using Commvault to backup Domino as well and would love to move to Veeam but we have DAOS enabled on our Domino Servers (attachments from emails are stored in a central database to reduce attachment deduplication).

Simply double clicking on a restored NSF file isn't going to give you access to the attachments as they are not stored in the NSF file.

I would imagine 90% of Domino Servers are running DAOS which means the solutions mentioned above aren't going to work in most situations. Has anyone come up with any workarounds?

[Andreas Neufert]

You can recover the DAOS files by searching for the missing files on the Server and use Veeam file restore for those.

https://www-10.lotus.com/ldd/dominowiki ... _databases
"To recover these, run the command:
tell daosmgr listnlo -o missingnlo.txt database_file_name.nsf
where “database_file_name.nsf” is the file name of the recovered database, and “missingnlo.txt” is the output text file containing a list of the NLOs that are referenced in the database but not available on the server. This list should be used to restore the required NLO files from the backup."

[zuldan]

@Andreas, thank you for the info. I will discuss with our Domino admins.

[serkantek]

Hi,

Sorry to bring this topic back from the dead. We did a POC with Veeam and Domino on one of our clustered servers with circular transactional logging enabled. Backup worked fine!
We have gone ahead now with Veeam as our backup solution and we would like to backup our production Domino server which as archived transactional logging enabled.

Does the transactional logging matter for Veeam backup? What is the best approach? How does incremental backups work with Domino and Veeam?
We only changed our production server to archived transcational logging so we could reduce the incremental backup size when backing up with Symantec Netbackup.

Any help/experiences will be appreciated.

Thanks

[Andreas Neufert]

Hi Serkan,
thanks for the request. The only way to backup a domino server is over the Domino backup API which uses a typical agent approach. With Veeam we do image level backups.
So what you can do is to bring domino in a pseudo consistent state (search this forums for the scripts) or shutdown one of your cluster servers and backup it.

Veeam can not handle the transaction logging or restore with transaction log roll forwards.

[serkantek]

Thanks Andreas,

I think we will change our Domino transaction logging style to circular and use the scripts to drop active sessions.
Thank you for your input.

[Spex]

We use pre/post thaw scripts for stop/start domino during backup with veeam (powershell).
In our environment the downtime for domino is between 1 and 2 minutes using this method.

The greatest problem we had was to have the correct execution rights on the server that was using uac.
We had to explicit give the user the right to start/stop the domino service.

[Andreas Neufert]

thanks for the feedback!

[mfilipp]

Dear all,
at this moment our Lotus Notes Server have a Independet Disk. So we cant take a Backup at the moment.
Now i will remove this Independent Disk and will convert it to a dependent Disk.
Is it possible to take a Backup if i will do this?

[Egor Yakovlev]

Hi Marcel,
check thread above for answers. Long story short, you can backup your server using Veeam Agent(without need of disk tricks), or using image-based VM backup by converting independent disk to normal.
/Cheers!

[h.ghali]

Hello everyone,

I am actually new to Veeam with Lotus Domino and we have a customer requiring backup for their Lotus Domino emails installed as a physical instance.
Can you please inform me which data should be collected from the customer in order to check how backup will be performed with Veeam?
Thank you in advance.

[HannesK]

Hello,
and welcome to the forums.

I think the main question is: "which time is okay for you to stop the Domino services to ensure consistent backup?" or "is it okay for you to do crash-consistent backup?"

Best regards,
Hannes