Agent-based backup of Windows, Linux, Max, AIX and Solaris machines.
Post Reply
DaveTech
Lurker
Posts: 2
Liked: never
Joined: Jun 03, 2024 8:44 am
Contact:

Synthetic full backup failure gets repository saturated

Post by DaveTech »

Hello there.

I have Veeam B&R 12 with a backup job set up to backup a Windows Server physical machine using an incremental + weekly synthetic backup scheme with a retention policy of 7 days.
All worked fine till few days ago, when the backup job stopped working because of full disk on the local repository.
After some troubleshooting, I realised that for some valid reason the backup job missed two synthetic backups in a row.
Those missing full backups basically caused the backup chain to grow up to 14+ restore points without any full backup in between. That stopped retention policy from being applied, and lately caused the backup job to fail because of missing free disk space. (Yeah, the machine being backed up has a high data change rate)
My only way out it was to delete the backups and start again from scratch.

How can this prevented from happening again?
Can't be that Veeam is not smart enough to detect this problem and automatically convert the daily backup job from incremental to a synthetic full, when the last synthetic backup failed.
karsten123
Service Provider
Posts: 571
Liked: 140 times
Joined: Apr 03, 2019 6:53 am
Full Name: Karsten Meja
Contact:

Re: Synthetic full backup failure gets repository saturated

Post by karsten123 » 1 person likes this post

size your repository accordingly to prevent this from happening.
DaveTech
Lurker
Posts: 2
Liked: never
Joined: Jun 03, 2024 8:44 am
Contact:

Re: Synthetic full backup failure gets repository saturated

Post by DaveTech »

This can't be the solution.
You cannot size the repository for the unpredictable. What if the weekly full backup keeps failing? Would you provide a repository of infinite size?
You size the repository based on backups that are expected to be on it, not for the ones that you do not expect.
An overdimensioned repository cannot be the answer to this problem.
And, in fact, this problem is not even taken into consideration by the repository size calculators provided by Veeam or by external third parties.

A proper solution would expect the backup agent to be aware of the result of the last backup, and to behave accordingly.

For example:
- If an active full backup failed, then the next backup should be an active full.
- If a synthetic full backup failed during the incremental backup generation, then the next backup should be a synthetic full.
- If a synthetic full backup failed during the conversion of the incremental backup to a full backup, then the conversion process should resume as soon as possible and before any further incremental backup.

I know that the solutions provided through the previous points were probably a bit too simplistic, but it should give an idea about the way to go.
The backup agent should be able to "react" to the failure of a full backup (or even an incremental backup).

I don't pretend this to be the standard behaviour (as I guess there are lot of infrastructures with overdimensioned repositories where the impact of extra backups on storage occupation is neglible), but there should at least be options for this.
This is especially critical for small environments (small and micro business) where building a very large repository is not an option, and cloud backup storage is not accessible.

As long as Veeam does not provide a proper solution, I guess I will try and write scripts to dynamically change the full backup schedule based on the result of the last backup. I don't like this solution, but I don't see alternatives.
karsten123
Service Provider
Posts: 571
Liked: 140 times
Joined: Apr 03, 2019 6:53 am
Full Name: Karsten Meja
Contact:

Re: Synthetic full backup failure gets repository saturated

Post by karsten123 » 1 person likes this post

lets talk numbers!
Used space? Size of the full? Daily change? Size of the repository? XFS/ReFS?
CBT drivers installed?
And what are these valid reasons, you are talking about?
Post Reply

Who is online

Users browsing this forum: No registered users and 22 guests