How "safe" is the new Linux proxy

[mkretzer]

Hello,

we had some major issues with Windows hotadd proxys in the past. Especially data inconsistencies because Windows mounted the volumes at backup time. For that reason we used NBD mode up until now.

Since V10 is there with linux hotadd proxy support i wonder how big the risk would be to use this new functionality in production now. I don't think linux will start to mount the hotadded windows NTFS but i don't know if that could not happen to Linux Filesystems/LVMs.

Are there any precautions we could take on the Linux proxy? Disable LVM scanning in some way?

Markus

[Gostev]

Hello!

It's impossible to provide a generic answer to this, simply because there's no such things as generic Linux. Instead, there is incredible number of various distributions, each with its own features and "smarts".

Now, the specific Linux distributions we tested do not have the automount logic similar to the one that makes Microsoft Windows cause issues you're talking about. Nor did we see any other potentially dangerous things done by the OS in our limited internal testing or during the beta program.

So, these distros can be considered "safe" from that perspective... that is, if you are willing to consider any brand new technology safe in principle. I mean, it took 15 years to discover the above issue in Windows, and only thanks to your "luck" I guess this fact alone says it all. Which is why I am always super conservative when it comes to any new tech, and my position is always "lets wait and see".

Thanks

[mkretzer]

I read somewhere that you are not using VDDK for this feature, correct? If you do use VDDK what is VMware recomending/requiring as proxy OS? I know from the other bug that using anything > W2016 is basically unsupported even with ESX 6.7.

[Gostev]

Correct, we don't use VDDK at all.

[mkretzer]

Do you plan to implement "VDDK-Less" Backup for Windows?

[Gostev]

It's been like that for years since v8 or something.

[mkretzer]

One second - why did we double check windows to esx (in other words VDDK) compatibly in the other case? Does that mean we can officially use 1903 as SAN and/or hotadd proxy? Support told us that it is no wonder we have issues since the proxy ran on a windows version too new for the VDDK of the host we want to back up!

[veremin]

Can you post your ticket number, so we can double check this statement internally? Thanks!

[Gostev]

Actually, I see no issues with this statement by support:
- VDDK is still used for SAN and NBD proxies normally.
- For hot add, we don't use VDDK as the transport engine, but we do use it for actual hot add operation.

But I would suggest creating new topic if we want to discuss Windows proxies further, as it makes it a worst off-topic for this particular thread

[mkretzer]

Ok but that makes sense. So there is a windows version dependency.

But nothing like that at all for linux?

[Gostev]

Not today at least. But, we are planning to add VDDK to Linux proxies in future releases, to enable support for other backup modes too.

[chris.jones]

It's impossible to provide a generic answer to this, simply because there's no such things as generic Linux. Instead, there is incredible number of various distributions, each with its own features and "smarts".

Now, the specific Linux distributions we tested do not have the automount logic similar to the one that makes Microsoft Windows cause issues you're talking about. Nor did we see any other potentially dangerous things done by the OS in our limited internal testing or during the beta program.

So, these distros can be considered "safe" from that perspective... that is, if you are willing to consider any brand new technology safe in principle. I mean, it took 15 years to discover the above issue in Windows, and only thanks to your "luck" I guess this fact alone says it all. Which is why I am always super conservative when it comes to any new tech, and my position is always "lets wait and see".

This is exactly why there needs (needed) to be a standardized Veeam Linux appliance to perform the proxy functions. Not only would it reduce the amount of variables and support calls, it would also eliminate the need for yet another Windows license.

[Gostev]

How is this different or better than using one of the tested and officially supported Linux distros from the System Requirements?

[mkretzer]

For us customers it is nice to have someONE to blame. Look at the situation with the RW mounted Windows volumes in our other case - you tell me that it is solved now with V10 - but the basic issues does still not seem to be solved from microsoft side - it basically took months until there is a real solution.

Now if the same happens with Linux it will be interesting, especially with distributions with slow release cycles like Debian (which we like very much).

[NTmatter]

This is exactly why there needs (needed) to be a standardized Veeam Linux appliance to perform the proxy functions.

It would actually be pretty cool to deploy Veeam Proxies in the same manner as the FLR appliances. Just specify a name and network settings, in addition to the regular proxy settings, then deploy the VM. No need to go through the Managed Servers setup.

[mkretzer]

Another argument for appliances coming from Veeam: You could automatically update these appliances when they are not in use by Veeam/deactivate them before update and so on. In theory this is all possible with standart linux systems but it needs some work. We do automatic updates at night time of our linux systems which we cannot simply do with Veeam proxys as there are always some backups running. Right now that would mean we either would not auto-patch Veeam proxys at all or we would have to develop some external logic which deactivates proxys, waits until backups are finished and then would patch the systems.

[Gostev]

It would actually be pretty cool to deploy Veeam Proxies in the same manner as the FLR appliances. Just specify a name and network settings, in addition to the regular proxy settings, then deploy the VM. No need to go through the Managed Servers setup.

These FLR helper appliances have been a huge security concern for many of our larger customers, so we certainly did not want to go this route, not to be eaten alive we're actually working on getting rid of them!

[colohost]

I'm curious what security concerns people have expressed about the FLR's? They would normally be spun up on the same network/VLAN as the Veeam install, and only exist for the duration of the restore job, along with having the ability configure the password, which can be changed as needed. Such a network should already have significant security precautions protecting it given Veeam B&R being on it and able to talk to the vSphere hosts, vCenter, etc.

In any case, are there pointers to threads / documentation on what the FLR replacement might be? We use them heavily with automation scripts.

[Gostev]

Too many too list, and I disagree with most anyway but then again, there are security folks who get paid to be... security folks! And this makes them a tough, often unreasonable crowd.

I guess the simplest one that is the hardest to argue is "they have a policy that restrict any pre-built 3rd party appliances on their network", period. Basically, not even something for a deep dive discussion, just plain NO. They don't want to deal with anything they didn't create and don't fully control.

We did not share any information on the replacement yet. But no worries, we will still keep FLR helper appliances as an option. There's still at least one valid use case to use them, even if a corner case. So, I don't ever see them removed completely.