Standalone backup agents for Linux, Mac, AIX & Solaris workloads on-premises or in the public cloud
Post Reply
tacioandrade
Enthusiast
Posts: 29
Liked: 3 times
Joined: Nov 17, 2016 2:04 am
Full Name: Tácio Andrade
Contact:

Run command before backup

Post by tacioandrade »

Good evening, some time here there is forum some comment on how to execute commands before and after running a Job on the agent. I would like to know if it is possible to create a Job by the manager in dialog and execute it after mounting an Amazon S3 pool via s3fs before running the job and after that remove it.
If yes, would anyone have a material on how to do the Job execution separately to put me a bash script?

Thanks for the help.
nielsengelen
Product Manager
Posts: 5619
Liked: 1177 times
Joined: Jul 15, 2013 11:09 am
Full Name: Niels Engelen
Contact:

Re: Run command before backup

Post by nielsengelen » 1 person likes this post

You can create a bash script which does the mount and then via veeamconfig command line starts the job. Add this to crontab and you are good to go.

First make a mount directory, in this example /s3mnt and point this one as the target for the job.

I don't know how to mount the S3 part correctly but based on http://tecadmin.net/mount-s3-bucket-cen ... sing-s3fs/ the script would look like this:

Code: Select all

mkdir /tmp/cache
chmod 777 /tmp/cache /s3mnt
s3fs -o use_cache=/tmp/cache mybackup /s3mnt
veeamconfig job start --name BackupJob
Make the bash executable chmod +x backup.sh and add it in a crontab to run.

This should get you started for now with the BETA2.
Personal blog: https://foonet.be
GitHub: https://github.com/nielsengelen
abrewerton
Novice
Posts: 4
Liked: 1 time
Joined: Nov 29, 2016 6:41 pm
Contact:

Re: Run command before backup

Post by abrewerton »

Is there a way to run a command before the backup runs and then run the backup and then run a command after the backup has completed. I would like to backup a MySQL database so I thought I would stop the MySQL service and the service for the application that runs against the MySQL database. Then I would run the backup and then start the MySQL database and the application service after the backup had completed.
tacioandrade
Enthusiast
Posts: 29
Liked: 3 times
Joined: Nov 17, 2016 2:04 am
Full Name: Tácio Andrade
Contact:

Re: Run command before backup

Post by tacioandrade »

vmniels wrote:You can create a bash script which does the mount and then via veeamconfig command line starts the job. Add this to crontab and you are good to go.

First make a mount directory, in this example /s3mnt and point this one as the target for the job.

I don't know how to mount the S3 part correctly but based on http://tecadmin.net/mount-s3-bucket-cen ... sing-s3fs/ the script would look like this:

Code: Select all

mkdir /tmp/cache
chmod 777 /tmp/cache /s3mnt
s3fs -o use_cache=/tmp/cache mybackup /s3mnt
veeamconfig job start --name BackupJob
Make the bash executable chmod +x backup.sh and add it in a crontab to run.

This should get you started for now with the BETA2.
Thanks a lot, the shell script part I already have experience, my question was how to run the job and if the job created by the "manager" would work.

I will do the tests on the weekend with direct backup on the S3 for example to see if it works well.
abrewerton wrote:Is there a way to run a command before the backup runs and then run the backup and then run a command after the backup has completed. I would like to backup a MySQL database so I thought I would stop the MySQL service and the service for the application that runs against the MySQL database. Then I would run the backup and then start the MySQL database and the application service after the backup had completed.
A great option too, run mysqldump right before the backup, today I use a script in crontab to generate the dump about 30 minutes before executing the backups with Bacula.
nielsengelen
Product Manager
Posts: 5619
Liked: 1177 times
Joined: Jul 15, 2013 11:09 am
Full Name: Niels Engelen
Contact:

Re: Run command before backup

Post by nielsengelen » 1 person likes this post

Do keep in mind that we also have the following options when creating a job with 'veeamconfig job create'

Code: Select all

 --postthaw <value>                - Post-thaw command (executed after snapshot creation)
  --prejob <value>                  - Pre-job command (executed on job start)
  --postjob <value>                 - Post-job command (executed on job finish)
These will be fully implemented in GA normally and will solve your problems. Just run the script before and after the job which does the steps for mysql (stop, mysqldump, start)

More info can be found by running:

Code: Select all

veeamconfig job create
Personal blog: https://foonet.be
GitHub: https://github.com/nielsengelen
tacioandrade
Enthusiast
Posts: 29
Liked: 3 times
Joined: Nov 17, 2016 2:04 am
Full Name: Tácio Andrade
Contact:

Re: Run command before backup

Post by tacioandrade »

vmniels wrote:Do keep in mind that we also have the following options when creating a job with 'veeamconfig job create'

Code: Select all

 --postthaw <value>                - Post-thaw command (executed after snapshot creation)
  --prejob <value>                  - Pre-job command (executed on job start)
  --postjob <value>                 - Post-job command (executed on job finish)
These will be fully implemented in GA normally and will solve your problems. Just run the script before and after the job which does the steps for mysql (stop, mysqldump, start)

More info can be found by running:

Code: Select all

veeamconfig job create
Nice knew that I had the option to create the job manually but I did not know why this option exists if there was already such a good manager in dialog, I now understand why.
This way it is even better to administer than via crontab, since sometimes the user instead of adding the script as root adds on its own user, which ends up making the environment a zone.
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Run command before backup

Post by PTide »

As a sidenote - personally I would double check if 777 permissions are really required in this case.

@abrewerton,
Is there a way to run a command before the backup runs and then run the backup and then run a command after the backup has completed. I would like to backup a MySQL database so I thought I would stop the MySQL service and the service for the application that runs against the MySQL database. Then I would run the backup and then start the MySQL database and the application service after the backup had completed.
There is no need to stop the entire service to backup the machine running the database - you can use pre-freeze / post-thaw scripts to put the db into read-only state during snapshot by issuing “FLUSH TABLES WITH READ LOCK” and then release the db after snapshot is taken. If you want to just dump the database wihtout backing up the whole machine then you should probably stick with mysql native tool.

Thanks
abrewerton
Novice
Posts: 4
Liked: 1 time
Joined: Nov 29, 2016 6:41 pm
Contact:

Re: Run command before backup

Post by abrewerton » 1 person likes this post

I was stopping the entire service because I do not have access to any passwords for the database. The vendor will not give those to me but the vendor will give me root access. I figured if I shutdown the vendors application and then turned off the MySQL service that all of the data would be cleanly written to disk. I could then run the backup through veeam and then restart the MySQL database and the application service. There is not a need for the application and database to be running 24/7. I found how to add the pre-freeze and post-thaw commands and verified that the services are being stopped and started successfully. Would stopping the application service then stopping the MySQL service then running the backup then starting the MySQL service and then starting the application service most likely get a consistent backup of the database. All of the data for the application is stored in the MySQL database.
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Run command before backup

Post by PTide »

The vendor will not give those to me but the vendor will give me root access
Honestly speaking that seems weird to me. AFAIR host system root access allows you to create your own mysql user and even reset mysql root password. Anyway I'm glad that things work smooth for you. Btw, have you tried to restore form the backup and perform database check?
abrewerton
Novice
Posts: 4
Liked: 1 time
Joined: Nov 29, 2016 6:41 pm
Contact:

Re: Run command before backup

Post by abrewerton »

Being that I do have root access, I probably could create my own MySQL user or reset the MySQL password but the vendor said they would not give root access to the database so I was trying to work within the requested boundaries. I have not had the time to test a full restore on a separate system. Being that I am working on a system already in production, I am trying to be as careful as possible to guarantee that I do not destroy production data.
tacioandrade
Enthusiast
Posts: 29
Liked: 3 times
Joined: Nov 17, 2016 2:04 am
Full Name: Tácio Andrade
Contact:

Re: Run command before backup

Post by tacioandrade » 1 person likes this post

I recommend that you do not do this, interrupt mysql depending on the transaction being made in the bank can corrupt it in the long run. It is best to do as PTide spoke and create another mysql user to back up via mysqldump.

I know how these software support companies do not give the DBMS password for fear of bullshit, but things like stopping the DBMS service can bring you problems in the future (although it works in the short and medium term).

If it does not want to give you a root access, it does not have problems, so for him to create a user with enough permission just to perform the backup of the database, I have done this in my clients, in the backup scripts I no longer place the root user and Yes a user with no power to update or insert, only select and another that is necessary to perform backup.
abrewerton
Novice
Posts: 4
Liked: 1 time
Joined: Nov 29, 2016 6:41 pm
Contact:

Re: Run command before backup

Post by abrewerton »

How is stopping the service any different than when the service is stopped when the computer is shutting down? How would that cause corruption?
nielsengelen
Product Manager
Posts: 5619
Liked: 1177 times
Joined: Jul 15, 2013 11:09 am
Full Name: Niels Engelen
Contact:

Re: Run command before backup

Post by nielsengelen »

If you normally do a proper shutdown of the server the service will also be halted proper so the impact is low.
Personal blog: https://foonet.be
GitHub: https://github.com/nielsengelen
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Run command before backup

Post by PTide »

Although mysqld should normally flush anything what is pending when it receives a shutdown signal, I don't think such a practice should be really used to ensure transactionally consistent backup. I agree that dirty hacks exist for reason, but having to shutdown the service in order to make a backup while having a root access is somewhat nonsense.
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 15 guests