Run command before backup

Backup agent for Linux servers and workstations on-premises or in the public cloud

Run command before backup

Veeam Logoby tacioandrade » Mon Nov 28, 2016 11:20 pm

Good evening, some time here there is forum some comment on how to execute commands before and after running a Job on the agent. I would like to know if it is possible to create a Job by the manager in dialog and execute it after mounting an Amazon S3 pool via s3fs before running the job and after that remove it.
If yes, would anyone have a material on how to do the Job execution separately to put me a bash script?

Thanks for the help.
tacioandrade
Enthusiast
 
Posts: 29
Liked: 3 times
Joined: Thu Nov 17, 2016 2:04 am
Full Name: Tácio Andrade

Re: Run command before backup

Veeam Logoby vmniels » Tue Nov 29, 2016 1:37 pm 1 person likes this post

You can create a bash script which does the mount and then via veeamconfig command line starts the job. Add this to crontab and you are good to go.

First make a mount directory, in this example /s3mnt and point this one as the target for the job.

I don't know how to mount the S3 part correctly but based on http://tecadmin.net/mount-s3-bucket-cen ... sing-s3fs/ the script would look like this:
Code: Select all
mkdir /tmp/cache
chmod 777 /tmp/cache /s3mnt
s3fs -o use_cache=/tmp/cache mybackup /s3mnt
veeamconfig job start --name BackupJob


Make the bash executable chmod +x backup.sh and add it in a crontab to run.

This should get you started for now with the BETA2.
VCP-DCV
Veeam Certified Architect (VMCA)
http://foonet.be
vmniels
Veeam Software
 
Posts: 1583
Liked: 349 times
Joined: Mon Jul 15, 2013 11:09 am
Full Name: Niels Engelen

Re: Run command before backup

Veeam Logoby abrewerton » Tue Nov 29, 2016 6:48 pm

Is there a way to run a command before the backup runs and then run the backup and then run a command after the backup has completed. I would like to backup a MySQL database so I thought I would stop the MySQL service and the service for the application that runs against the MySQL database. Then I would run the backup and then start the MySQL database and the application service after the backup had completed.
abrewerton
Novice
 
Posts: 4
Liked: 1 time
Joined: Tue Nov 29, 2016 6:41 pm

Re: Run command before backup

Veeam Logoby tacioandrade » Tue Nov 29, 2016 10:25 pm

vmniels wrote:You can create a bash script which does the mount and then via veeamconfig command line starts the job. Add this to crontab and you are good to go.

First make a mount directory, in this example /s3mnt and point this one as the target for the job.

I don't know how to mount the S3 part correctly but based on http://tecadmin.net/mount-s3-bucket-cen ... sing-s3fs/ the script would look like this:
Code: Select all
mkdir /tmp/cache
chmod 777 /tmp/cache /s3mnt
s3fs -o use_cache=/tmp/cache mybackup /s3mnt
veeamconfig job start --name BackupJob


Make the bash executable chmod +x backup.sh and add it in a crontab to run.

This should get you started for now with the BETA2.


Thanks a lot, the shell script part I already have experience, my question was how to run the job and if the job created by the "manager" would work.

I will do the tests on the weekend with direct backup on the S3 for example to see if it works well.

abrewerton wrote:Is there a way to run a command before the backup runs and then run the backup and then run a command after the backup has completed. I would like to backup a MySQL database so I thought I would stop the MySQL service and the service for the application that runs against the MySQL database. Then I would run the backup and then start the MySQL database and the application service after the backup had completed.


A great option too, run mysqldump right before the backup, today I use a script in crontab to generate the dump about 30 minutes before executing the backups with Bacula.
tacioandrade
Enthusiast
 
Posts: 29
Liked: 3 times
Joined: Thu Nov 17, 2016 2:04 am
Full Name: Tácio Andrade

Re: Run command before backup

Veeam Logoby vmniels » Tue Nov 29, 2016 10:27 pm 1 person likes this post

Do keep in mind that we also have the following options when creating a job with 'veeamconfig job create'

Code: Select all
 --postthaw <value>                - Post-thaw command (executed after snapshot creation)
  --prejob <value>                  - Pre-job command (executed on job start)
  --postjob <value>                 - Post-job command (executed on job finish)


These will be fully implemented in GA normally and will solve your problems. Just run the script before and after the job which does the steps for mysql (stop, mysqldump, start)

More info can be found by running:
Code: Select all
veeamconfig job create
VCP-DCV
Veeam Certified Architect (VMCA)
http://foonet.be
vmniels
Veeam Software
 
Posts: 1583
Liked: 349 times
Joined: Mon Jul 15, 2013 11:09 am
Full Name: Niels Engelen

Re: Run command before backup

Veeam Logoby tacioandrade » Tue Nov 29, 2016 11:05 pm

vmniels wrote:Do keep in mind that we also have the following options when creating a job with 'veeamconfig job create'

Code: Select all
 --postthaw <value>                - Post-thaw command (executed after snapshot creation)
  --prejob <value>                  - Pre-job command (executed on job start)
  --postjob <value>                 - Post-job command (executed on job finish)


These will be fully implemented in GA normally and will solve your problems. Just run the script before and after the job which does the steps for mysql (stop, mysqldump, start)

More info can be found by running:
Code: Select all
veeamconfig job create


Nice knew that I had the option to create the job manually but I did not know why this option exists if there was already such a good manager in dialog, I now understand why.
This way it is even better to administer than via crontab, since sometimes the user instead of adding the script as root adds on its own user, which ends up making the environment a zone.
tacioandrade
Enthusiast
 
Posts: 29
Liked: 3 times
Joined: Thu Nov 17, 2016 2:04 am
Full Name: Tácio Andrade

Re: Run command before backup

Veeam Logoby PTide » Wed Nov 30, 2016 10:02 am

As a sidenote - personally I would double check if 777 permissions are really required in this case.

@abrewerton,
Is there a way to run a command before the backup runs and then run the backup and then run a command after the backup has completed. I would like to backup a MySQL database so I thought I would stop the MySQL service and the service for the application that runs against the MySQL database. Then I would run the backup and then start the MySQL database and the application service after the backup had completed.

There is no need to stop the entire service to backup the machine running the database - you can use pre-freeze / post-thaw scripts to put the db into read-only state during snapshot by issuing “FLUSH TABLES WITH READ LOCK” and then release the db after snapshot is taken. If you want to just dump the database wihtout backing up the whole machine then you should probably stick with mysql native tool.

Thanks
PTide
Veeam Software
 
Posts: 3140
Liked: 262 times
Joined: Tue May 19, 2015 1:46 pm

Re: Run command before backup

Veeam Logoby abrewerton » Fri Dec 09, 2016 3:26 pm 1 person likes this post

I was stopping the entire service because I do not have access to any passwords for the database. The vendor will not give those to me but the vendor will give me root access. I figured if I shutdown the vendors application and then turned off the MySQL service that all of the data would be cleanly written to disk. I could then run the backup through veeam and then restart the MySQL database and the application service. There is not a need for the application and database to be running 24/7. I found how to add the pre-freeze and post-thaw commands and verified that the services are being stopped and started successfully. Would stopping the application service then stopping the MySQL service then running the backup then starting the MySQL service and then starting the application service most likely get a consistent backup of the database. All of the data for the application is stored in the MySQL database.
abrewerton
Novice
 
Posts: 4
Liked: 1 time
Joined: Tue Nov 29, 2016 6:41 pm

Re: Run command before backup

Veeam Logoby PTide » Fri Dec 09, 2016 4:14 pm

The vendor will not give those to me but the vendor will give me root access
Honestly speaking that seems weird to me. AFAIR host system root access allows you to create your own mysql user and even reset mysql root password. Anyway I'm glad that things work smooth for you. Btw, have you tried to restore form the backup and perform database check?
PTide
Veeam Software
 
Posts: 3140
Liked: 262 times
Joined: Tue May 19, 2015 1:46 pm

Re: Run command before backup

Veeam Logoby abrewerton » Fri Dec 09, 2016 4:39 pm

Being that I do have root access, I probably could create my own MySQL user or reset the MySQL password but the vendor said they would not give root access to the database so I was trying to work within the requested boundaries. I have not had the time to test a full restore on a separate system. Being that I am working on a system already in production, I am trying to be as careful as possible to guarantee that I do not destroy production data.
abrewerton
Novice
 
Posts: 4
Liked: 1 time
Joined: Tue Nov 29, 2016 6:41 pm

Re: Run command before backup

Veeam Logoby tacioandrade » Sat Dec 10, 2016 2:55 am 1 person likes this post

I recommend that you do not do this, interrupt mysql depending on the transaction being made in the bank can corrupt it in the long run. It is best to do as PTide spoke and create another mysql user to back up via mysqldump.

I know how these software support companies do not give the DBMS password for fear of bullshit, but things like stopping the DBMS service can bring you problems in the future (although it works in the short and medium term).

If it does not want to give you a root access, it does not have problems, so for him to create a user with enough permission just to perform the backup of the database, I have done this in my clients, in the backup scripts I no longer place the root user and Yes a user with no power to update or insert, only select and another that is necessary to perform backup.
tacioandrade
Enthusiast
 
Posts: 29
Liked: 3 times
Joined: Thu Nov 17, 2016 2:04 am
Full Name: Tácio Andrade

Re: Run command before backup

Veeam Logoby abrewerton » Mon Dec 12, 2016 5:35 pm

How is stopping the service any different than when the service is stopped when the computer is shutting down? How would that cause corruption?
abrewerton
Novice
 
Posts: 4
Liked: 1 time
Joined: Tue Nov 29, 2016 6:41 pm

Re: Run command before backup

Veeam Logoby vmniels » Mon Dec 12, 2016 5:37 pm

If you normally do a proper shutdown of the server the service will also be halted proper so the impact is low.
VCP-DCV
Veeam Certified Architect (VMCA)
http://foonet.be
vmniels
Veeam Software
 
Posts: 1583
Liked: 349 times
Joined: Mon Jul 15, 2013 11:09 am
Full Name: Niels Engelen

Re: Run command before backup

Veeam Logoby PTide » Mon Dec 12, 2016 7:15 pm

Although mysqld should normally flush anything what is pending when it receives a shutdown signal, I don't think such a practice should be really used to ensure transactionally consistent backup. I agree that dirty hacks exist for reason, but having to shutdown the service in order to make a backup while having a root access is somewhat nonsense.
PTide
Veeam Software
 
Posts: 3140
Liked: 262 times
Joined: Tue May 19, 2015 1:46 pm


Return to Veeam Agent for Linux



Who is online

Users browsing this forum: No registered users and 3 guests