Comprehensive data protection for all workloads
Post Reply
DonZoomik
Service Provider
Posts: 368
Liked: 120 times
Joined: Nov 25, 2016 1:56 pm
Full Name: Mihkel Soomere
Contact:

Feature request: Autodistribution of VM fulls per day in job

Post by DonZoomik »

Automatic distribution of full backups over week/month in job

Business case:
We'd like to squeeze more data into out WS2012R2 based NTFS dedup enabled repository. Currently the issue is weekly active full backups that consume a lot of disk space and dedup takes a while to process them. During full backups repository tends to get full that causes chains to move to another disk, significantly worsening dedup efficiency.
While distributing VMs manually to different jobs (7 jobs, each one would have full on different workday) would be possible, it is pretty bad micromanagement. Currently we prefer to keep on job per cluster to pretty much automatically catch any new VMs and keep management overhead low.

I suppose Veeam could semiautomatically distribute full jobs over days. Eg
100 VMs in job / 7 days = ~14-15 fulls per day
100 VMs in job / 30 days = ~3-4 fulls per day
That'd be a good start that would cut the micro. Also we could reduce free space requirements on repositories, allowing more data to be backed up.

It would be even better if it could balance incoming data per repository extent per day.
For example a 100TB scale out repository with 3 extents (33TB each). Veeam would do it's best to equalize incoming data per repository to 33/7=~5TB per day. If extents had different sizes, it would scale daily data accordingly.
Veeam already has metadata to allow this
* Past VBK sizes for indication about full size
* Past VIB sizes for indication about per-day/run delta
* VMWare tools guest used disk space about new VMs
* VMWare used (thin) disk space if tools are not available
This would allow us to reduce required free space even more, again allowing more density.

For example daily VIBs would be ~1TB, this would leave 4TB for fulls that Veeam would select for full backup. This feature would probably require a new option, something like 'maximum restore points between fulls' as day of full backup can move back and forth depending on changes in environment (eg number of VMs in job changes, size on VBK/VIB changes...).
Day 1: Full on that 4TB big file server
Day 2: A few dozen 100GB VMs
Day 3: 4 1TB VMs
Etc...

Of course there are edge cases (change in VM size, balancing of data between extents, addition or removal of VMs, other jobs pointed at the same SOBR...) but even the first scenario would allow us to easily increase repository density while keeping maintenance requirements low.

I'm working on a script to semi-automatically manage this with per-VM jobs (to allow full to move back and forth per-VM without moving VM to another causing unintentional fulls) but it's pretty complex stuff that basically works against builtin logic. Something like that built-in would be wonderful.
KarmaKuma
Enthusiast
Posts: 32
Liked: 6 times
Joined: Feb 05, 2022 11:16 am
Contact:

Re: Feature request: Autodistribution of VM fulls per day in job

Post by KarmaKuma »

Sorry for digging up an old thred, but I was looking for this functionality, has this been somehow enabled by now?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Feature request: Autodistribution of VM fulls per day in job

Post by foggy »

Hi, no, there's no ability to auto-distribute fulls for VMs within a single job between the weekdays.
KarmaKuma
Enthusiast
Posts: 32
Liked: 6 times
Joined: Feb 05, 2022 11:16 am
Contact:

Re: Feature request: Autodistribution of VM fulls per day in job

Post by KarmaKuma »

closest thing would be to create "n" jobs, where "n" is the number or weekday I ant to distribute my fulls on?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Feature request: Autodistribution of VM fulls per day in job

Post by foggy »

Yes, you can manually distribute VMs between jobs and schedule fulls for different days.
Post Reply

Who is online

Users browsing this forum: Bing [Bot], DataDefender and 295 guests