Comprehensive data protection for all workloads
Post Reply
rhys.hammond
Veeam Software
Posts: 72
Liked: 15 times
Joined: Apr 07, 2013 10:36 pm
Full Name: Rhys Hammond
Location: Brisbane , Australia
Contact:

unable to add replica vms to cloud failover plan

Post by rhys.hammond »

Hey Everyone,

Just working on a project that involves a brand new Veeam Backup & Replication v9.5U3 configured with a Service Provider consuming CC-R.
Replication is working pretty well, we had a couple small wrinkles that needed to be ironed out but everything feels stable at the moment.

One issue that I cannot solve is that when we create a Cloud Failover Plan, we are unable to add any objects to the failover plan. See attached image.
I performed some basic troubleshooting to no avail so opened a support Case # 02446414

If I right-click a cloud replica VM I can 'Add to failover plan' and select 'New cloud failover plan' but if I try and add another cloud replica no luck. When I right click and expand the 'add to failover plan' the only option is to create a new failover plan.

Our next steps are to contact the Service Provider to troubleshoot on their end and collect logs from them but I wanted to post on the forums first to see if anyone else had seen this issue?
To me it feels like it could be a firewall issue but we know the replication to the SP is working (working after we enabled TCP/UDP for their required port).
The reason being is when we tested from a Veeam BnR Console that wasn't installed on the VBR server, we can't see any Cloud replicas at all.

Looking at the firewall logs we can't see anything being blocked though.

Cheers Rhys Hammond

Image
Veeam Certified Architect | Author of http://rhyshammond.com | Veeam Vanguard | vExpert
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: unable to add replica vms to cloud failover plan

Post by dellock6 »

I've never seen this issue before honestly. Just so we are sure everything is up to date, that both provider and client are at 9.5u3?
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
rhys.hammond
Veeam Software
Posts: 72
Liked: 15 times
Joined: Apr 07, 2013 10:36 pm
Full Name: Rhys Hammond
Location: Brisbane , Australia
Contact:

Re: unable to add replica vms to cloud failover plan

Post by rhys.hammond »

Hey Luca,

Yep, both tenant side and SP are both at 9.5u3
Veeam Certified Architect | Author of http://rhyshammond.com | Veeam Vanguard | vExpert
rhys.hammond
Veeam Software
Posts: 72
Liked: 15 times
Joined: Apr 07, 2013 10:36 pm
Full Name: Rhys Hammond
Location: Brisbane , Australia
Contact:

Re: unable to add replica vms to cloud failover plan

Post by rhys.hammond »

Veeam support has been escalated to the cloud team, will post an update when I know more.
Veeam Certified Architect | Author of http://rhyshammond.com | Veeam Vanguard | vExpert
n.zahid
Lurker
Posts: 1
Liked: never
Joined: Jul 07, 2017 10:32 am
Full Name: Nauman Zahid
Contact:

Re: unable to add replica vms to cloud failover plan

Post by n.zahid »

I have got the same issue and Veeam support is investigating. Is there any update on this yet ?
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: unable to add replica vms to cloud failover plan

Post by dellock6 »

Not that I'm aware, please keep both updated about what support will find out.

Thanks,
Luca
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
rhys.hammond
Veeam Software
Posts: 72
Liked: 15 times
Joined: Apr 07, 2013 10:36 pm
Full Name: Rhys Hammond
Location: Brisbane , Australia
Contact:

Re: unable to add replica vms to cloud failover plan

Post by rhys.hammond »

status of my open support ticket is currently: Research/Testing
Veeam Certified Architect | Author of http://rhyshammond.com | Veeam Vanguard | vExpert
rhys.hammond
Veeam Software
Posts: 72
Liked: 15 times
Joined: Apr 07, 2013 10:36 pm
Full Name: Rhys Hammond
Location: Brisbane , Australia
Contact:

Re: unable to add replica vms to cloud failover plan

Post by rhys.hammond »

just heard back from support, they found some orphaned entries on the service provider database that could be preventing us from creating the failover plan. (two job entry from the service provider database (for the tenant) that has no OIB's.)
We need to clean up this entry on the service provider database and tenant database and then try to create the failover plan again.

Will report back once cleaned and tested.
Veeam Certified Architect | Author of http://rhyshammond.com | Veeam Vanguard | vExpert
rhys.hammond
Veeam Software
Posts: 72
Liked: 15 times
Joined: Apr 07, 2013 10:36 pm
Full Name: Rhys Hammond
Location: Brisbane , Australia
Contact:

Re: unable to add replica vms to cloud failover plan

Post by rhys.hammond » 1 person likes this post

ok Veeam support were able to successfully resolve this issue. The root cause were few a backup ids with zero OIB. After we removed the orphaned entries from the provider database we were able to create the failover plan successfully.
In my case the only entries that were cleaned up were on the SP side, not the tenant.
Veeam Certified Architect | Author of http://rhyshammond.com | Veeam Vanguard | vExpert
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: unable to add replica vms to cloud failover plan

Post by dellock6 »

Thanks Rhys for updating the thread, and glad it has been fixed.

Luca
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
rhys.hammond
Veeam Software
Posts: 72
Liked: 15 times
Joined: Apr 07, 2013 10:36 pm
Full Name: Rhys Hammond
Location: Brisbane , Australia
Contact:

Re: unable to add replica vms to cloud failover plan

Post by rhys.hammond »

This issue has reoccured after the SP and tenant upgraded to 9.5u3a
Have opened a new support ticket with Veeam to resolve
Veeam Certified Architect | Author of http://rhyshammond.com | Veeam Vanguard | vExpert
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: unable to add replica vms to cloud failover plan

Post by veremin »

Kindly, let us know the support ticket ID, so we can follow the investigation. Thanks.
rhys.hammond
Veeam Software
Posts: 72
Liked: 15 times
Joined: Apr 07, 2013 10:36 pm
Full Name: Rhys Hammond
Location: Brisbane , Australia
Contact:

Re: unable to add replica vms to cloud failover plan

Post by rhys.hammond »

The case# was 03141448,

Same resolution steps as last time but support shared how to troubleshoot these myself for next time.
For those that are interested, from the SP side you run the following query to find a stale job entry and then delete it.

Code: Select all

select T1.id as backup_id, T1.job_name, t2.id as point_id, T3.id as OIB_id, t3.point_id from dbo.[Backup.Model.Backups] as t1 full outer join dbo.[Backup.Model.Points] as t2 on t1.id = t2.backup_id left outer join dbo.[Backup.Model.OIBs] as t3 on t2.id = t3. point_id where t1.id in ( select backup_id from [Backup.Model.ViCloudTenantBackups] where [quota_id] in ( select id from [Backup.Model.ViHardwareQuotas] where tenantId = (select id from dbo.tenants where name = [i]'%TenantName%[/i]'))) and t3.id is null
Replace %TenantName% with the tenant name in CC

The output should be something like

Code: Select all

064bfb1c-7a1e-42c9-bad4-40629f6c4048f2ca3a17-18b5-4690-82f8-530834453b3a%VMNAME%
%VMNAME% will depend on the VM object that is stale.

Query to delete stale entry:

Code: Select all

EXEC [dbo].[Backup.Model.DeleteBackupChildEntities] '064BFB1C-7A1E-42C9-BAD4-40629F6C4048'
EXEC [dbo].[Backup.Model.DeleteBackup] '064BFB1C-7A1E-42C9-BAD4-40629F6C4048'
Output:
1 Row affected
Veeam Certified Architect | Author of http://rhyshammond.com | Veeam Vanguard | vExpert
Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 162 guests