Painful V12 Cumulative Updates

[mkaec]

After V12 was released, there was 12.0.0.1420_20230223. That was OK. It was 122 MB. Then came 12.0.0.1420_20230412 and 12.0.0.1420_20230718, 3.5 GB each. Now the patches are not ISOs like the original release. They are EXE files. First I download the patch as a ZIP file. Then I extract it inside the VM and run it. But, what happens? It tells me to wait while it extracts again. So, there's a lot waiting involved here. And this process has the potential to increase the VM's dynamically expanding disk by 7 GB.

I'd much prefer if the patches were distributed as ISOs like the original releases. Then I could mount them at the host level and them run in the VM from the virtual DVD drive.

[DanielJ]

+1 on this. It gets even worse when the VBR server is physical like a couple of all-in-one instances I have upgraded. Both had a system disk with 8-9 GB free and it was not enough, and there was no way of telling the installer to use the storage disk (with several TB free) for the extensive amount of work space. I could only upgrade those by moving stuff in AppData away and linking to it with junction, then move it back with the services down.

[Gostev]

Right, and these are not the only issues with the current approach that need to be addressed. The corresponding work is being done in 12a to improve the patching experience significantly.

[popjls]

Do these CU's update/upgrade any Enterprise Manager website components? Being that Defender hates the install process, I'm wondering if we need to disable/offboard Defender every time an update occurs now? Disabling Defender does not seem to work otherwise I'd just do that. Rather not break the install.

[Gostev]

Yes, they do. So if you have EM installed on a separate machine, you will need to run the update there as well.

[terry_zolinskiTC]

As someone who is heavily reliant on AWS, limiting the amount of white space on our systems is crucial. I agree with everything said by other posters, Veeam needs to figure out how to handle this temp space requirement better.

To upgrade to the cumulative update in July I had to add 10 GB.

I currently have 20 GB free on C:\ and this latest upgrade needs 35 free??? This is absolutely ridiculous, considering after the upgrade is done I will have almost 30 GB free.

During the last upgrade I did notice that Veeam puts a copy of the entire ISO on the C:\, so that's a huge problem. Maybe I misconstrued what happened there, but I remember deleting a 13 GB file off of the C:\ to free up space to run the upgrade.

And this is just a test environment...

[Gostev]

The truth is that (perhaps outside of AWS) the requirement of 35GB of free disk space cannot honestly be considered a major issue... it's more of a joke in the world where even my home PC has twice as much RAM alone! And a 2TB SSD...

So with that in mind, please appreciate the fact that it's hard for us PMs to make solving this a priority for R&D vs. having them focus on features and improvements that really add major value to the product.

[terry_zolinskiTC]

For the record, comparing your 2 TB SSD in your home PC as a cost qualifier to an Enterprise Organization is not even close to relevant, whether or not AWS is in the picture. Have you ever purchased a 2 TB NetApp SSD for an Enterprise class storage array? The cost per terabyte goes up dramatically.

I understand your points surrounding R&D and making the product better is critical. At the same time though, you can't spread yourself too thin and fall short in the areas that got you here in the first place.

I just finished my upgrade successfully, here is my finding.

First running setup.exe I am told I need 32.59 GB free to perform the install. I have 20 free.
Run the install anyway, my disk space free increases to 27 GB free. The installer freed up it's own space.
The install ran fine, my disk space never going below 25 GB free (5 GB more than I had initially).
I am currently sitting here with VBR upgraded to 12.1 with 29.7 GB free (5 GB less than the installer said I needed to perform the install).

And after further review the installer has cleaned up the remains of the VeeamBackup&Replication_12.0.0.1420_P20230412_20230606 install in July.

These two files were left over after the previous upgrade (7.6 GB):
C:\Program Files\Veeam\Backup and Replication\Backup\Updates\veeam_backup_12.0.0.1420_CumulativePatch20230412.exe
C:\Program Files\Veeam\Backup and Replication\Backup\Updates\veeam_backup_12.0.0.1420_CumulativePatch20230718.exe

Those files were removed by this installation.

So, the installer immediately said to add 15 GB space to accommodate the upgrade, yet it freed up more space than it needed post upgrade.
The July update failed because of disk space and I had to monitor the install with TreeSize to find it was copying those cumulative patch files to the C:\, causing the upgrade to fail for files that were not required.

I have used other vendor's products that were buggy in production. One of the issues was their development team did not use a fully up to date code repo during a major release and caused major production issues for us. We worked with their engineering team to sort out that product, it was a nightmare. I would hate to trade in that solution for another one like it. We chose Veeam because it works, so don't fault us that want you to focus on QA to prevent your customers from being in bad situations. I have been there, we replaced that solution with Veeam...

[terry_zolinskiTC]

At the end of the day, I do agree that cloud computing has caused organizations to trend toward bad practices in favor of cost savings. I don't agree with it, but that's the job I do. Tech savvy guys like me MUST be cost conscious, so trivial issues like disk space, when you factor in the scale of 10,000 users, becomes a big deal. Back in the day, turn on thin provisioning and forget about it, but as Veeam moves into the cloud space, you need to forget the old-world thinking. AWS makes money off of thin provisioning ideology, Veeam does not. Customers get hosed at the end of the day either way, so with the AWS deck stacked against us, it's nice having Vendors like Veeam still on the customer's side.

[Gostev]

Thanks for the detailed outline of the actual disk space usage. Perhaps the estimations UI is giving are incorrect in the end. I've asked the responsible PM to discuss these observations with the corresponding QA team. Appreciate you flagging this!

[scott.anderson]

+1 for realistic disk requirements.

Veeam VBR seems to have a lot of "spare" files floating around now. There are zip files everywhere. Which is fine if they are needed during install, but why leave them lying around, filling up the C: drive so that the next upgrade can't run because its run out of space? Having all the components installed is nice, but if your not using the Mac Agent etc having the space used up and updates etc also lying around on the C: drive is not helpful.

Tier one hardware vendors - Boot drives on internal SSD's quite often 128Gb until very very recently.

So if your upgrading Dell, Lenovo, HPE etc in place VBR server trying to find the 37Gb that the VBR 12.1 installer thinks it needs to start can become "troublesome" very quickly on a server that isn't that old, but has had a couple of VBR upgrades and the normal Windows updates.

So whether its a cloud solution (where space optimisation is how you save significant $$$) or a physical datacentre solution on Tier 1 vendor hardware, the reality is that your C: drive isn't going to be large and trying to add space is probably not going to be an easy option.

If files are not needed after install, please can we get them cleaned out properly.

If we could select the location for agent packages to be installed, that would also be useful.