Running Status not accurately reflected

[sbk]

Running on Veeam 9.5u3
Case # 02454216

All my Replication jobs show a status of “99% completed”, even though they are not actually at 99% completed. If you select the running job and look at the status in the detail window at the bottom of the screen, it shows its actual status.
I know there has been a bug where you have to manually refresh the screen in order to get an accurate idea of what’s actually running vs. what the screen is saying but this is different. I would also like to note that I am surprised the refresh bug wasn’t addressed in 9.5u3.
Anyone else come across this issue?

[Vitaliy S.]

Hi Shawn,

Do you possibly have Veeam ONE deployed to compare the jobs states? Just trying to narrow down the issue either to the GUI or wrong status in the database.

Thanks!

[sbk]

I do have an instance of Veeam ONE running, but I cannot see the running percent completed via. Veeam ONE (not sure if I am supposed to be able too). I can just see that the jobs is running, and the duration / transferred data, etc.
Another noteworthy item is that this only seems to be an issue on the replication jobs. The normal backup to disk jobs report an accurate percentage as expected

[foggy]

Hi Shawn, have you demonstrated this behavior to our support engineers?

[sbk]

Not yet, I provided some logs over the weekend, expecting a follow-up today.

[sbk]

I am still working with support but essentially it sounds like this is a well known bug that is in the works to be fixed in version 10.
Official response:
"After researching further this appears to be a known issue when sourcing a replica or backup copy from a restore point created by backup from storage snapshot.
Essentially in 9.5 it was changed how the size of the job is calculated; from the restore point rather than the infrastructure. This gives a boost in job performance.
In cases of a backup object created by storage snapshot, the information for "total size" is not included in the restore point's auxiliary data stored by Veeam.
With no data on the total size, the job progress calculation jumps straight to 99% and stays that way to the very end of the job."

My issue with this is that the actual job percentage is being displayed correctly in the job details, you just have to highlight/click each running job to see that detail. So the information IS available, seems like the job status is simply not pulling from the correct location. We typically have 20+ replications running at any given moment so this "bug" makes it difficult to keep track of progress at a glance. At the end of the day this is all the result of upgrading to Enterprise plus as we wanted to utilize the Nimble / Veeam snapshot integration.

[foggy]

Thanks for the update, Shawn, much appreciated.

[sbk]

Just wanted to provide a follow-up to this post.
Working with Veeam Support, they were able to create a hot-fix that has resolved the issue. The percentage is now updating as expected!

[sbk]

*Another update on this issue... Apparently the most recent Veeam Backup and Replication patch broke this again (9.5 update 3a) and I'm being told by support that not only will they NOT provide a hot fix this time, they don't know if it will even be included in the next patch (expected to be released near the end of the year) and that it may remain broken until version 10. I waited a long time to update to 3a, and only did so as we are thinking about finally updating to vCenter 6.7 and would need to be on the latest Veeam release in order to support that. Now that I have upgraded there is no turning back just short of losing all current restore points for backups and replications if I wanted to downgrade back to a version that actually works. So if you happen to come across this post - beware of upgrading if you want to see your replication status accurately reflected.

*Current case number 03238354.
It's disappointing that this has been broken all year, and the fix was not included in the 9.5 3a update. Sounds like the new policy is that NO hot fixes will come out anymore for customer issues - you will have to wait until the next official release.

[foggy]

As far as I can see from the case, this fix is going to be included in Update 4 that is coming out soon.