delayed VBR Console refresh

[BackupBytesTim]

Curious, Gostev, is the issue, and presumably the hotfix, on the VBR console or VBR server? If you know for sure at this point. I only have 1 VBR server, it's a Cloud Connect VBR server, with enough activity to actually notice the problem so I'm curious where the hotfix might be applied to. Our customers' individual VBR servers are relatively small and I'm rarely connected directly to them anyways, so a server hotfix would be easy enough to apply, but only when we have planned maintenance time. A console hotfix I could apply immediately for testing, but if it actually fixes the problem would then need to be applied to multiple console installations to cover all our staff.

[Gostev]

Looks like it's on the server, in the SQL script.

[BackupBytesTim]

Just checking in for a quick update, making sure nothing's been forgotten since it's been another 12 days now. Still working on that fix?

I know it's more of a "band-aid" solution, but I'd be good with some console-side fix that simply gets it to periodically refresh, performing whatever normally happens when I press F5, just automatically every like 10-30 seconds. Or at least something to refresh when it first opens, to save that added step of pressing F5 when I open it. I assume something like that would be fairly easy to implement, although not necessarily a long-term fix, since it's just automating a function that presently can be performed manually by pressing a keyboard button.

[chris.childerhose]

Glad to hear the issue is known and looking forward to a fix. I don't mind F5 but would be nice not to use it.

[BackupBytesTim]

I'm glad to hear there's definitely more than just one or two people experiencing the issue. For a while I was assuming it was just me, but since it seems much more widespread than that, hopefully a fix won't take too long.

I'll admit pressing F5 isn't like, a challenge, but I still find it unacceptable that I have that added task of pressing a refresh button so often, especially if I just leave the console window on the screen and go do other things, having to keep going over to it and refreshing it is irritating. Honestly tempted to write a PowerShell script myself to simulate pressing F5 periodically. I've done it before with more complex procedures to automate app installations, something as simple as pressing F5 every 10 seconds shouldn't be a challenge, but as anything would be useless once an actual fix is released, I keep hoping we'll get a fix soon so I haven't bothered yet.

[pmichelli]

+1 here

Sometimes Just clicking on different objects in the VBR console produces a delay. The other day for example I was watching a backup job run and when I would click between the different VMs to see how fast they were going, there would be a pause of up to 5 seconds before the VBR console would unfreeze itself. Doesn't happen every click but happens every few clicks consistently.

[mcz]

Anton, is postgres affected as well?

[chris.childerhose]

From what I have seen, PostgreSQL shows similar behaviour but I am going to test more in my lab.

[HannesK]

yes, same on Postgres (my environment is on Postgres)

[pmichelli]

I am also on Postgres with this problem. My V11 was on SQL. I upgraded it to V12 then migrated it to a new instance running Postgres

[kaffeine]

We're experiencing the same. In our case it affects our backup jobs from SQL Server VMs which include their T-LOGs as well (they're backup up every few minutes).

For example, let's assume we have an incremental backup from 5 SQL VMs in the middle of the day:
# throughout the whole morning the 5 corresponding T-LOG backup jobs are always visible on the console the SQl VMs, all good
# midday comes and an incremental snapshot is made from all VMs
# after that the T-LOG backups are resumed and now we're seeing 10 backup jobs on the console: the 5 T-LOG backups that were running before the snapshot (they should be gone from the console now) and the same 5 T-LOG backups that are resumed after the snapshot

Pressing F5 we the console is cleaned from the old job runs and we're seeing only the 5 currently running.

It's only a minor/cosmetic issue, but leads to a some confusion every now and then.

Btw we faced this V11 and now on the most recent V12 patch as well.

Regards

[BackupBytesTim]

That sounds like the same primary issue, with any sort of jobs, the console view shows the jobs and usually updates their status, but if they complete and go away, they still show on the list even if the same job/task starts again, the last one will still sit there in the view.

Similarly we've also seen issues where the console displays unpredictable information when it is first opened, it doesn't display current accurate information, but also doesn't display the information that was being displayed at the time it was last closed. Which is what lead to my conclusion that the "refresh" feature must be refreshing something on the server and directing the console to then retrieve new information, else I would think the console is retrieving new information when it opens, but in this case it appears the console does retrieve new information from the server, however the information on the server is inaccurate so the console is retrieving inaccurate information.

Hopefully we get some sort of fix soon, it is one of those little things, but I've been dealing with it since V11 and sometimes I still forget to refresh it before I glance at it and be like "well, that's wrong, something's broken" and then I'll go look for what's broken, just to remember "oh wait, gotta refresh the console".

[without_xp]

If it helps any I can certainly collect relevant logs to upload in a support case if desired, but I'd need direction as to which logs are desired.

We have a potential solution to this problem, but we can't definitively test it because we don't have a re-confirmation in our lab.
Could I kindly ask you to open a support case with a link to this forum thread so that we give you a fix within it and you check if it solves the problem?

[mcz]

I'll open a ticket as well to support this case. Edit: case # 06145585

[BackupBytesTim]

I have also opened a support case.
#06146436

[NickKulkarni]

@BackupBytesTim
This has been a bug since V11, I was lead to believe it would be fixed in the next release.
Sadly this doesn't seem to be the case.

[mcz]

okay, received the fix today and the first impression is that it might have fixed it. For the moment it's a bit too early to make that conclusion but the first signs are good.

[chris.childerhose]

So is the fix a patch or DB fix? Just curious going to open a case today for the fix.

[mcz]

it is a DB fix (stored procedure)

[chris.childerhose]

Thanks for the update. Will be interesting to see what it does.

[mcz]

ok, first I thought the issue was fixed an I wannted to write that here when just in that moment the job-list refreshed itself and showed old orphaned jobs two days ago. Also very interesting: I've done a replica failover and then an undo later and now I see the "active (0)" under the root node "replicas". Normally that one would disappear if there are no jobs left, but that's here still not the case. Maybe another hint to find the root cause?

[HannesK]

the way goes via support and might require a higher log level or you wait until developers went through all videos I sent to them where F5 solved the problem (that can take some time)

[mcz]

Yes, I'm still working with support on it, just wannted to notify everyone who's checking that thread to let them know that it hasn't fixed it - at least not for me

[BackupBytesTim]

I'm still working with support as well, they asked for more logs, which I provided, but haven't gotten a reply yet. Someone mentioned to me most of you in Europe may have holidays for a few days here in a row, so I'm not too worried about the delay. Just mentioning that I'm still working with support on it.

[BackupBytesTim]

We have a potential solution to this problem, but we can't definitively test it because we don't have a re-confirmation in our lab.
Could I kindly ask you to open a support case with a link to this forum thread so that we give you a fix within it and you check if it solves the problem?

Could I get someone to poke the support team and get them to skip over whatever process they're doing before just giving me the fix? It's been 12 days now since the fix has supposedly been available and, despite my linking to this thread in my initial opening of the support case, I've still yet to get support to actually send me anything. The case has been through a few support reps, gotten additional logs provided, been escalated, and now it's just sitting there.

Case #06146436

I was really expecting that linking to this thread when I opened the case would get them to just look at the thread and see there's an issue and send me the potential fix, not go through this process that's taken almost 2 weeks now.

[Mildur]

Hi Tim

I asked support management to have a look.

Best,
Fabian

[mcz]

ok, so today my engineer asked me for extended UI logging + a process dump of the affected process during failure occurrence. I've sent the requested data and now it should have been passed to the devs. Just for information, so that everyone knows that the investigation is still going on.

[mcz]

ok, so log collection wasn't enough to find the root cause. There is a suspicion that the CDP jobs trigger this. Has anyone seen this bug without any CDP jobs configured?
In any case, we are working on another log collection to dig even deeper.

[chris.childerhose]

Yes, I have seen it without CDP jobs configured. I don't have any configured now, and I see it happening in PostgreSQL.

[mcz]

Thanks Chris, will pass this to the support team as well.