Backing up postgres db within Docker volume

[MatHanson]

I am designing a backup strategy for a Docker environment and am having difficulty finding information on how this will behave, so I was hoping from some input from anyone that understands these processes more than I!

The environment is as follows -
Veeam Backup and Replication v10
Linux Veeam Agent - 4.0.0.1961-1.el7
DockerCE 3:19.03.12-3.el7
Postgres 12.3

Here's a drawing of the solution -



The way that I am hoping that this will behave is that the Veeam Agent will use application-aware processing to communicate with postgres and get the database into a stable and consistent state to take a backup, and then it will take a volume-level backup of the "/db" volume.

Firstly, am I correct in my thinking of this process? And secondly, will this process freeze database usage while the backup is occurring?

[HannesK]

Hello,
and welcome to the forums.

The Veeam agent can detect the Postgres database on the "same platform". It cannot "look inside" hundreds or thousands of containers and check whether somewhere a Postgres database is running inside.

As a workaround, you could use pre / post-scripts that have knowledge about the containers.

Best regards,
Hannes

[MatHanson]

Hi Hannes,

Thanks for the welcome and for the reply, appreciate it.

This is where I was unsure of the behaviour of the processing - the host only houses one postgres container and this instance is directly accessible via port 5432 of the host. I assumed that the Veeam agent would be communicating over this port and should be able to talk to the db as if it was directly installed on the host alongside the Veeam agent. Is my assumption incorrect?

[PTide]

Hi @MatHanson,

Actually, it might work (although we never tested it hence cannot be officially supported in the current version).

However, there is another snag - in order to back up pgsql, VAl relies on its own snapshot capabilities. Technically, a nested container will be included in the snapshot too, but that hasn't been extensively tested too.

Let me check with our dev team and I'll get back you shortly

Thanks!

[MatHanson]

Hi @PTide,

Thanks for the information. I actually ran a test on this last night and the job did actually finish with "Require successful processing" checked. So I assume this means that the Veeam agent was able to communicate with the containerised postgres instance, as I now have a backup of the directory which holds the docker volume. I am going to do some further testing on this, but it looks promising! Would love to hear any feedback that the devs might be able to give. Thanks again!

[HannesK]

for successful processing, the logs should say something like "PostgeSQL checkpoint has been triggered successfully"

like on this picture

[MatHanson]

Hi Hannes,

So this looks like the processing didn't work then?


I have checked the job logs and can't find any mention of postgres, is there anything in particular that I can look for in the logs to figure out why it has failed the processing?

[HannesK]

I did not talk to developers, but from my point of view, it cannot see it. That means, there is no "failure".

There is nothing in the logs, because from the agent perspective, it does not exist. Things that do not exist are not logged.

[MatHanson]

Hi Hannes,

I understand that, I am trying to understand why it can't see it - as I don't understand how Veeam is attempting to connect to the database. Does Veeam communicate with postgres via TCP/IP or via a unix-socket? If it's via unix-socket, then I can just use a bind-mount to expose the socket from within the container to /var/lib/postgresql on the host - I just can't find any information from Veeam explaining just how it is trying to look for this db instance?

[PTide]

Veeam connects to the database via libpq library.

That is, if the library is present on the host (Docker) and is aware that it needs to contact a certain DB via TCP/IP on a certain port and IP address, and that port and address is reachable from the host, then it might work.

On the other hand, if you can tolerate a couple of minutes of transactions not getting into the backup, you could probably skip the AAIP completely. Given the nature of pgsql, you could just snapshot the whole things without even caring about creating a checkpoint (it is created automatically quite frequently anyway).

Thanks!

[MatHanson]

Hi PTide,

Thanks for that info... I didn't realise that it was safe to backup a running postgres database without AAIP. We can totally tolerate not having the current transactions in the backup, I just didn't realise this was the only drawback to not using AAIP. Thanks again, my job here is done!