Feature Request - Update Cache/Repository

[BackupBytesTim]

Inspired by a recent discussion on the VCSP forums: post486826.html#p486826

Particularly useful for office locations with slow internet download speeds, it could be quite helpful to have VBR servers be able to act as a centralized update cache or repository. So any time an update is released by Veeam the VBR server can download it, so the update can be retrieved automatically by nearby configured devices or manually from a folder, that way updates can already be available locally when an IT admin is ready to install things, whether manually or using an automated process. I would say this should include all Veeam components, and include versions that may be newer and incompatible with management from the relevant VBR server, especially the update to the VBR server itself.

If I were designing it, I'd probably want both the ability to manually choose Veeam components to download and store updates for, as well as an automatic option to select components based on connected/managed devices and what software is actually in use. It could also be useful to choose how many recent releases of each component should be kept locally, as I can see cases when not everyone will want to jump to the latest version, but may still like the ability to use a local update cache.

[HannesK]

Hello,

retrieved automatically by nearby configured devices

could you maybe explain which devices / components that would be? Do you have multiple VBR servers per branch office?

And just to be clear: Is the request about the Service Provider Console or multiple standalone VBR servers?

Best regards,
Hannes

[BackupBytesTim]

We do not have multiple VBR servers at a single location no, but I mean for any Veeam components at all, VBR servers, console, agents, any component that may need to be updated individually at any point.

The idea is to help with locations with a slow internet download speed, so when updates become available they're automatically downloaded by a local server, so that when it's time to install them they're already available on a device at the location and don't need to be downloaded from Veeam's servers at update time.

[HannesK]

already available on a device at the location and don't need to be downloaded from Veeam's servers at update time.

that sounds like a scenario that is only applicable with the Service Provider Console (or I still don't understand the design here)

In a VBR environment it looks like this today
- one download to the VBR server -> upgrade everything on the VBR server
- managed server updates to branch offices are small. If that causes a problem, please provide more information on the bandwidth you have and the problems you see. I heard of 2MBit/s per branch office and less working fine
- agents: there is the distribution server that can be used per location. Agents are larger, here it makes sense to have distribution caches, yes
- console: agree, that one is heavy if it has to be distributed centrally (e.g. 50 consoles in one remote location). Do you have such a design? (I know what it means to live on low bandwidth )

[BackupBytesTim]

For us the main benefit would be with agent updates, but it would also be helpful for the sake of having more convenient local access to the files in the event of a manual installation, whether for an update or a new install.

As we do use the VSPC for management pretty exclusively, but still have VBR servers present in many locations that also have agents, I'm curious if that single download to the VBR server for local distribution can be used to upgrade agents when the agents are NOT managed by the VBR server?

We do not have that many consoles in a single location, no. And what we do have is actually all virtual machines in a data center with regular work done via Remote Desktop, so the consideration is more for the sake of customer-side components.

I will say, while I think it would be a beneficial feature to have, it is a bit more hypothetically beneficial for us as we have very few customers where such a thing would be useful. Though I imagine there's plenty of other people where it would be more beneficial. I wanted to suggest it since the idea sort of came up in the other thread, but if I were picking things for the development team to spend time on, there's plenty else I'd rather see added first.

[HannesK]

I'm curious if that single download to the VBR server for local distribution can be used to upgrade agents when the agents are NOT managed by the VBR server

hmm, how would unmanaged agents find any type of caching system? Sound like a configurable download URL could do that in combination with a customer-hosted webserver. But not sure whether that's what you like to do.

[BackupBytesTim]

I suppose I should clarify, in my case I mean that they're not managed by the VBR server, but are managed by the VSPC server at a remote location. But yes, if I were throwing something together I'd probably have what amounts to a basic web server built-in to whatever component hosted the cache and have that server's address be configured as a primary address for retrieving updates from. Optimally this could be configured from the managing VSPC server, which is important, having the feature configurable locally could be useful for some people but as we're a service provider that provides only Veeam to most of our customers, we don't have the ability to easily modify local-only Veeam settings, of which I keep finding there's many of.

I suppose I should also add, my idea was that adding the feature to the existing VBR server would be a simple way to go, and I imagine few places that don't have a VBR server would have enough of any Veeam software to bother with such an update cache at all, however it could very easily just be a separate component. For an unrelated project I had created a PowerShell script that did basically what I'm talking about. My thoughts is much along the lines of a software I made for a different company, I had a "configuration program" that would install and configure numerous different apps from numerous different companies on a computer, both for first-time setup of the software and for updates to the software in the future. To simplify things I had created a separate PowerShell script that acted as a sort of cache server, the configuration program would download all the required files from said cache server, which hypothetically had everything stored locally, and if any required file was not stored locally it would download the file from the official location and then send it to the computer that requested it.

It could make sense to have the cache server be a separate program entirely for the benefit of people not using a VBR server, but I imagine for Veeam's development process that would be more complicated to do than just adding a feature to the existing VBR server.

[HannesK]

okay, but you are asking in the VBR forum for something that you already asked in the VSPC forum as far as I understood? So my focus here is VBR and the solution is the distribution service, which already exists.

For VSPC: If there are 100 agents in one location and there is a VAW update, then the Master Agent of that location can download the new version and the other 99 agents would get it from there. I'm still not sure where VBR comes in here.

A picture of your scenario could help.

[BackupBytesTim]

I understand the purpose of the Master Agent, but my request here is for a similar feature for other components, not just the agents, but also VBR servers and anything else that may be present in the customer's local environment. We use the VSPC for remote management, yes, but there are still different components installed at the customer's location, VBR servers for backing up virtual machines, agents for backing up physical workstations and some host servers, as well as dedicated repository servers with their own Veeam components installed locally at the customer's office.

My idea here is to have a single central VBR server, or other software component of Veeam infrastructure, which is located within the customer's office network, that can centrally download updates for all these components when Veeam publishes the updates, and then have all those different components on all the different physical systems at the customer's office retrieve updates from that central VBR server when the update is initiated, rather than the current behavior which is for each individual component to communicate with our VSPC server or Veeam directly to retrieve the updates when an update is initiated.

So sort of two separate things, having a single place within an office network where updates come in to the office network from whatever outside source, our VSPC server or Veeam's servers, and then also the ability for that central update cache server to download all the updates as soon as Veeam publishes them, even if the device they go on hasn't initiated the update yet, so that later on when the update is initiated, be it hours or days or weeks later, the update is already available on the local network.

Also, correct me if I'm wrong, but my understanding is that our current scenario where we have many VSPC linked Management Agents which all can hypothetically communicate independently with both our VSPC server and Veeam's servers, means that the "Master Agent" does not function as a central download source for the agents at an office, and they'll all just download the update independently. Is that incorrect? Assuming it's correct then unless there's a way to change that behavior so everything downloads updates from the local Master Agent, then that doesn't actual answer the "central repository" idea, as my idea is not to solve a connection problem, but just to alleviate long update times resulting from updating many devices at a single location with slow download speeds.

[HannesK]

but also VBR servers and anything else that may be present in the customer's local environment

that's what VBR does if you update. It updates all components that are attached to it. No download from the internet / VSPC.

means that the "Master Agent" does not function as a central download source for the agents at an office, and they'll all just download the update independently.

I'm not sure, that I can follow. That's the opposite of what the first sentence of the link about the Master Agent says. But please let's stick with VBR on the VBR forums and put everything else in the VSPC forums to avoid confusion.