Automatic distribution of full backups over week/month in job
We'd like to squeeze more data into out WS2012R2 based NTFS dedup enabled repository. Currently the issue is weekly active full backups that consume a lot of disk space and dedup takes a while to process them. During full backups repository tends to get full that causes chains to move to another disk, significantly worsening dedup efficiency.
While distributing VMs manually to different jobs (7 jobs, each one would have full on different workday) would be possible, it is pretty bad micromanagement. Currently we prefer to keep on job per cluster to pretty much automatically catch any new VMs and keep management overhead low.
I suppose Veeam could semiautomatically distribute full jobs over days. Eg
100 VMs in job / 7 days = ~14-15 fulls per day
100 VMs in job / 30 days = ~3-4 fulls per day
That'd be a good start that would cut the micro. Also we could reduce free space requirements on repositories, allowing more data to be backed up.
It would be even better if it could balance incoming data per repository extent per day.
For example a 100TB scale out repository with 3 extents (33TB each). Veeam would do it's best to equalize incoming data per repository to 33/7=~5TB per day. If extents had different sizes, it would scale daily data accordingly.
Veeam already has metadata to allow this
* Past VBK sizes for indication about full size
* Past VIB sizes for indication about per-day/run delta
* VMWare tools guest used disk space about new VMs
* VMWare used (thin) disk space if tools are not available
This would allow us to reduce required free space even more, again allowing more density.
For example daily VIBs would be ~1TB, this would leave 4TB for fulls that Veeam would select for full backup. This feature would probably require a new option, something like 'maximum restore points between fulls' as day of full backup can move back and forth depending on changes in environment (eg number of VMs in job changes, size on VBK/VIB changes...).
Day 1: Full on that 4TB big file server
Day 2: A few dozen 100GB VMs
Day 3: 4 1TB VMs
Of course there are edge cases (change in VM size, balancing of data between extents, addition or removal of VMs, other jobs pointed at the same SOBR...) but even the first scenario would allow us to easily increase repository density while keeping maintenance requirements low.
I'm working on a script to semi-automatically manage this with per-VM jobs (to allow full to move back and forth per-VM without moving VM to another causing unintentional fulls) but it's pretty complex stuff that basically works against builtin logic. Something like that built-in would be wonderful.