Storage policies - the idea

[DanielJ]

Let me present an idea that might simplify some things regarding repositories, retention and jobs: storage policies.

What is a storage policy? It is a set of rules that define where, how and how long to store data. Instead of assigning a repository to a job, you now assign a storage policy. The retention settings are moved from the jobs to the storage policy. The storage policy can contain a single rule (save data on this repository for this many days), or it can contain multiple rules that chain to each other.

Examples of things that could be expressed using storage poilcy rules:
- Save the data for x days on repository 1 (the simplest possible case).
- Save the data for x days on any one of a list of repositories (VBR selects the one with the most free space).
- Save the data for x days on repository 1 and for y days on repository 2. The data is written to both repositories at the same time (if possible - otherwise queued and then copied).
- Save the data for x days on repository 1, then move it to repository 2 and save it there for y days.
- Save the data for x days on any one of this list of repositories, then move it to any one of this other list of repositories and save it there for y days, then move it to repository 3 and save it there for z days.

Encryption settings are moved from the jobs to the storage policy (and as much other settings as possible that has to do with how backup data is stored/maintained).
Repositories are defined like before. GFS settings, like daily retention, are done in the storage policy and works like before on all backups written according to that policy.
If different retention levels are needed, just create another storage policy. After a storage policy is changed, all backups written by that policy are subjected to the changes immediately by a maintenance job (applying new retention, moving backups to a new repository, etc).

Upgrading current jobs should be fairly simple and automated. VBR will create storage policies that match the current repository and retention settings in the jobs, so that if the user does not do anything else than upgrading, everything should continue to work as before.

Pros:
- No need for copy jobs. The storage policy can be defined to save any number of copies (as long as there are enough different repositories). Things that were complex or impossible before (copy of a copy of a copy, or similar) are now much easier to implement.
- No need for SOBR. The same capabilities can be achieved with storage policy rules, but with less complexity and limitations. Individual repositories can still be targeted.
- Data can be migrated/offloaded freely, in a chain fully defined by the user, not bound by predefined tiers.
- If you want a single retention for all your jobs and all data encrypted (a common use case), you can now set it in one place.

Cons:
- Some complex setups could be difficult or impossible to convert automatically.
- ?

[HannesK]

Hello,
thanks for the ideas. I guess the main con is, that it means changing the entire product for hundreds of thousands of users

The ideas in general look valid and from what I hear, there are products that work more or less the way you suggest. The main question is, which problems do you try to solve? We are aware of some limitations of the job-based approach vs. policy, but we also work on reducing these limitations.

Best regards,
Hannes

[DanielJ]

There is no big problem to solve, just thinking out loud. But replacing two big and complex things (copy jobs and SOBR) with something that could do both in a simpler, less limited way should be an advantage for the product both as it is now and to build on for the future. And it would still be job-based, just with some settings moved out of jobs to where they make more sense.

[tyler.jurgens]

I guess the main con is, that it means changing the entire product for hundreds of thousands of users

Veeam 13 feature confirmed!