Host-based backup of VMware vSphere VMs.
wasc
Service Provider
Posts: 24
Liked: never
Joined: Jan 11, 2012 4:22 pm
Full Name: Alex
Contact:

Backing up a Windows Failover Cluster with Shared Storage?

Post by wasc »

I'm wondering if Veeam can be used to backup a Windows Failover Cluster using Shared Storage - for example a SQL Failover Cluster. I know the answer has always historically been a no, as Veeam can only backup something in vmware that you can snapshot. However, As I understand it, Veeam can backup a Virtual mode RDM, and I also believe that as of VMware 5.5, a virtual mode RDM is supported for a Windows 2012 Failover Cluster?

Therefore, would I be right in thinking I could create a Failover Cluster in my Primary Site, and back this up to my secondary site using veeam, where the backup contains ALL the data, and not just the System Drive (C drive)?
Vitaliy S.
VP, Product Management
Posts: 27375
Liked: 2799 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Backing up a Windows Failover Cluster with Shared Storag

Post by Vitaliy S. » 1 person likes this post

Hello Alex,

According to VMware online documentation Failover Clustering and Microsoft Cluster Service is supported for vRDMs, but it doesn't tell you anything about snapshotting capability of such configuration.

As far as I remember MS Failover Cluster requires bus sharing, which in turn prevents you from doing snapshots, so VM-image backup is still not possible.

BTW, do you have a VM configured in such a way to confirm this assumption?

Thank you!
wasc
Service Provider
Posts: 24
Liked: never
Joined: Jan 11, 2012 4:22 pm
Full Name: Alex
Contact:

Re: Backing up a Windows Failover Cluster with Shared Storag

Post by wasc »

Hi Vitaliy,

Thanks for the response. I haven't set this up in a lab yet, so I don't know if this will work - I was hoping someone had tried it already. If I do find the time to set this up in a lab, i'll be sure to update the post with my results.
Vitaliy S.
VP, Product Management
Posts: 27375
Liked: 2799 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Backing up a Windows Failover Cluster with Shared Storag

Post by Vitaliy S. »

Alex,

In addition to my previous post, I have just found a pretty informative blog post on MS Failover Cluster feature and its support in latest vSphere release. Its author is still missing VM-image snapshots, so I'm afraid the solution above is not going to fly:
There are a few things that would be nice to have in VMware vSphere for Windows Failover Clusters and I hope these things are included in future releases:
...
Support for vADP style backups
...
Thanks!
wasc
Service Provider
Posts: 24
Liked: never
Joined: Jan 11, 2012 4:22 pm
Full Name: Alex
Contact:

Re: Backing up a Windows Failover Cluster with Shared Storag

Post by wasc »

Ah, well, that pretty much finishes that possibility then :)

Shame - for a minute there I got excited. The ability to backup or replicate a clustered machine would be the best new feature that could be added to the vsphere product line.
Vitaliy S.
VP, Product Management
Posts: 27375
Liked: 2799 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Backing up a Windows Failover Cluster with Shared Storag

Post by Vitaliy S. »

Well....they keep enhancing this feature every release... :)
dellock6
VeeaMVP
Posts: 6166
Liked: 1971 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Backing up a Windows Failover Cluster with Shared Storag

Post by dellock6 »

Alex, and what about instead the new non-shared clusters, like Exchange DAG or the new SQL 2012 AlwaysOn? This is something in my opinion is going to make shared MS clusters obsolete, they are much more flexible, easier to configure both in application and for the underlying storage, at the only expense of more storage; since storage prices are always reducing, this is no more such a huge problem.

no shared storage, no lock on shared vmdk or RDM, so snapshots are possible ;)

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
trafsta
Enthusiast
Posts: 45
Liked: 2 times
Joined: Apr 10, 2012 5:50 pm
Contact:

[MERGED] 2-node SQL 2014 Standard Clusters and Backups w/ Ve

Post by trafsta »

I am looking to implement a 2-node SQL Server 2014 Standard cluster (AlwaysOn FCI is what they call these days I believe). It would have been great to be able to go with AlwaysOn Availability Groups, but alas there is no way our medium sized business could afford the SQL Enterprise licenses... that said, I've searched via Google and via this forum for the best way to set this up while still supporting snapshots and backups through Veeam, but I'm still a bit confused as to which path to take despite all the reading I have done, so I thought I should post a new thread (feel free to merge this to another existing thread that I may have somehow missed...).

Here is a further rundown of our environment:

VMware ESXi 5.1 hosts
Dell EqualLogic SANs for storage
Veeam B&R 7.0.0.871 (going to 8.x as soon as it's released)
2 x SQL 2014 Standard servers looking to be clustered in a Veeam-compatible way (Windows Server 2012 R2)

I basically want to have the capability to reboot the SQL Servers one by one for monthly MS Windows and SQL updates without taking down all the servers and applications that rely on the SQL DBs running on this SQL 2014 environment. A manual failover method would be sufficient (could update the 'passive' node, reboot, manual failover active to passive, then update the active node, reboot, and manual failover back - all hopefully without affecting any of the servers/apps connected to the SQL DBs [hoping the downtime during failover is just a few seconds...]). Our existing single SQL 2008 server is pretty much 24/7 nowadays so it is always a pain to schedule downtime windows to do updates and reboots, so I'm hoping AlwaysOn FCI or some other Veeam-compatible clustering will get around this. So far my reading has pointed to using 2014's AlwaysOn FCI's w/ shared storage, but what I'm not 100% positive about is what type of shared storage to use (SAN, SMB, etc) that will still allow Veeam to properly backup the SQL DBs using snapshots. I don't like the idea of using SMB for shared storage either considering that will just be another single point of failure for update reboots... so I'm stuck with shared storage on the SAN... but what do I need to use, shared .vmdk's? Or something else?

Also I'm not sure if there will be any changes to the way B&R 8.x works with regards to AlwaysOn FCI's and if it will be able to support backing them up better than 7.x currently can.

Any input on this topic would be greatly appreciated.
foggy
Veeam Software
Posts: 21138
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backing up a Windows Failover Cluster with Shared Storag

Post by foggy »

Unfortunately, no type of shared storage (be it either in-guest iSCSI, pRDM, vRDM, SMB) supports snapshotting.

The upcoming Veeam B&R v8 though adds full support for Availability Groups, including both backup and restore.
trafsta
Enthusiast
Posts: 45
Liked: 2 times
Joined: Apr 10, 2012 5:50 pm
Contact:

Re: Backing up a Windows Failover Cluster with Shared Storag

Post by trafsta »

foggy wrote:Unfortunately, no type of shared storage (be it either in-guest iSCSI, pRDM, vRDM, SMB) supports snapshotting.

The upcoming Veeam B&R v8 though adds full support for Availability Groups, including both backup and restore.
Thanks foggy. I assume that means I'd have to go with SQL 2014 Enterprise in order to do AlwaysOn Availability Groups.

Such a shame. Perhaps when the next SQL release occurs (2014 R2 or 2017, or whatever they are going to call it) they'll then make the AOAG available in the Standard edition...

I guess for now I'm just going to have to use a standalone install of SQL 2014 Standard and live with scheduling MS updates/reboots. :(

Thanks for your response.
verre
Enthusiast
Posts: 67
Liked: never
Joined: Jun 30, 2014 1:51 pm
Full Name: Valentino Erre
Contact:

[MERGED] Backup a MSCS cluster

Post by verre »

Hi everybody, I have two virtual machine in MSCS cluster with a pRDM for the cluster quorum. I already know there is no chance to backup the pRDM, anyway I was wondering if is it possible to backup (and restore) the virtual machine and its vmdk bypassing the RDM disk (actually I needn't to backup the quorum)
Vitaliy S.
VP, Product Management
Posts: 27375
Liked: 2799 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Backup a MSCS cluster

Post by Vitaliy S. »

If you're using bus sharing, then VM backup would not be possible, see this topic for more info > Ideal way to setup windows cluster with shared storage

As to other VM configurations, then please be aware that Veeam B&R automatically excludes all unsupported disk types from backups.
verre
Enthusiast
Posts: 67
Liked: never
Joined: Jun 30, 2014 1:51 pm
Full Name: Valentino Erre
Contact:

Re: Backup a MSCS cluster

Post by verre »

Vitaliy S. wrote:If you're using bus sharing, then VM backup would not be possible, see this topic for more info > Ideal way to setup windows cluster with shared storage

As to other VM configurations, then please be aware that Veeam B&R automatically excludes all unsupported disk types from backups.
Thank you Vitality, sorry for insisting, but it must be clarified that if a vmdk can't be backupped it is not correct to say that the machine backup would not be possible!
I've just tried to do it: a virtual machine with 2 vmdk: the first one is "snapshot allowed" and the second is a "indipendent persistent": I excluded the second vmdk during the backup job creation!

((( I don't understand why, although I excluded the second vmdk, the job finished with the warning "Disk 'VEEAM-test_1.vmdk' has been skipped due to unsupported type (independent)" but this is only a warning, indeed the restore works correctly )))

Therefore my assumption: if I need to backup a Win machine in MSCS cluster it is possible to backup its C:\ disk but not the quorum (due to impossibility to backup a pRDM).

Is there any reasons why do I need to backup the quorum? I don't think so... but I was asking you any opinion....
Vitaliy S.
VP, Product Management
Posts: 27375
Liked: 2799 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Backup a MSCS cluster

Post by Vitaliy S. »

Snapshots are always taken for the entire VM, only unsupported disk types are excluded by VMware automatically. Exclusions that are available in the Veeam backup console affect the backup job, but not the snapshot creation.
verre
Enthusiast
Posts: 67
Liked: never
Joined: Jun 30, 2014 1:51 pm
Full Name: Valentino Erre
Contact:

Re: Backup a MSCS cluster

Post by verre »

Vitaliy S. wrote:Snapshots are always taken for the entire VM, only unsupported disk types are excluded by VMware automatically. Exclusions that are available in the Veeam backup console affect the backup job, but not the snapshot creation.
Sure! Indeed this explain the warning ... but nevertheless this does not detract that the backup works! The warning appears only for the first full backup, but not for the subsequent incremental backups... therefore it is possible to backup a vm even if that has independent disks (such as mscs cluster).
foggy
Veeam Software
Posts: 21138
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup a MSCS cluster

Post by foggy »

That means that your MSCS cluster is configured using in-guest iSCSI initiators, not VMware SCSI bus sharing Vitaliy was referring to.
Dave338
Enthusiast
Posts: 40
Liked: 5 times
Joined: Jan 27, 2015 12:21 pm
Full Name: David
Contact:

Re: Backup a MSCS cluster

Post by Dave338 »

If you cannot make a snapshot at vmware level, you cannotmake a backup of the machine.

Also this can not be possible because there is a controller ser to bus sharing, requisite for mscs...so again not possible.

If you want a backup of this machines you need a operating system level backup, like acronis or symantec system recovery, like in a physical machine.

Regards
veremin
Product Manager
Posts: 20405
Liked: 2298 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup a MSCS cluster

Post by veremin » 1 person likes this post

Or Endpoint Backup Free.
pkelly_sts
Veteran
Posts: 600
Liked: 66 times
Joined: Jun 13, 2013 10:08 am
Full Name: Paul Kelly
Contact:

Re: Backup a MSCS cluster

Post by pkelly_sts »

v.Eremin wrote:Or Endpoint Backup Free.
That caught my attention!

Is it really feasible to use Endpoint Backup Free for MSCS clusters? I haven't looked into Endpoint at all as I considered it a "client" thing which doesn't interest us but I might want to take a closer look at it now if it's feasible in such a config...
Vitaliy S.
VP, Product Management
Posts: 27375
Liked: 2799 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Backup a MSCS cluster

Post by Vitaliy S. »

Endpoint Backup is a "client thing". Do not expect full-blown protection to be available for MSCS clusters, as Endpoint target is laptops and PCs. Take a look a this topic for more info > Veeam Endpoint Backup Free
ChrisGundry
Veteran
Posts: 259
Liked: 40 times
Joined: Aug 26, 2015 2:56 pm
Full Name: Chris Gundry
Contact:

[MERGED]: MSCS backups

Post by ChrisGundry »

Firstly, happy new year to all! Hope that none of you had any horrible Christmas callouts!

Anyway, the reason for my post is to talk about MSCS and Veeam backups.

Who is using Veeam to backup GUEST MSCS servers and in what way? Are you running Hyper-V/VMware as the hypervisor? What disk configuration? What is MSCS doing for you, file servers, SQL, DHCP etc?

The research I have done (and checked with Veeam themselves) seems to say that its basically not possible to backup a GUEST MSCS node with Veeam B&R due to issues with snapshots and shared disks/bus sharing etc.

The only options as I can see them are:
1. Use Veeam Endpoint backup from within the guest OS - Is anyone using this for MSCS? I know it is more of a client thing... What is Veeam's support view on this scenario?
2. I can use Windows backup from within the guest OS
3. Use in guest iSCSI initiators to do the disk sharing and use Veeam B&R to backup the guest itself, but it won't be able to backup the guest iSCSI disks and will have to try and back them up some other way, perhaps storage snapshots...?

Many thanks for any input from people using MSCS and Veeam!

Chris
widmerkarl
Expert
Posts: 122
Liked: 29 times
Joined: Jan 06, 2015 10:03 am
Full Name: Karl Widmer
Location: Switzerland
Contact:

Re: MSCS backups

Post by widmerkarl »

Hi Chris,

Thanks, also to you a happy new year!

Just to clarify, what do you mean with MCSC? Microsoft Cluster Services? Because this and a Master Degree in Computer Science are the only two things which i can find... Please help us out if it's NOT Microsoft Cluster Services.

I'm backing up different servers with Veeam, from Microsoft SQL, Exchange, fileservers, Active Directory, Linux Appliances (Security, Spam Scanning etc) and had no issues. For sure, there are some trial an error procedures until everything works, but now the backups are running fine.

I think it doesn't matter what you are backing up. I think it depends how your are backing it up.

If there are Microsoft virtual machines, e.g. on Hyper-V or vSphere, you can use the VSS feature withing Veeam (Application-Aware image processing, http://helpcenter.veeam.com/backup/80/v ... on_vm.html). With that feature you will get a clean and consistent backup of your virtual machine.

Don't use the VMware Tools Quiescence feature if it's not really necessary (http://helpcenter.veeam.com/backup/80/v ... cence.html). At least in my environment it causes the VM's to freeze for about a half a minute or so. Imagine that on a fileserver serving network shares / network drives to the users, or on a terminalserver, the users get a fine disconnect, and i get fine phone support calls.
Karl Widmer
IT System Engineer

vExpert 2017-2024
VMware VCP-DCV 2023 / VCA6-DCV / VCA5-DCV / VCA5-Cloud / VMUG Leader
Former Veeam Vanguard / VMCE v9 / VMTSP v9 / VMSP v9
Personal blog: https://www.driftar.ch
Twitter: @widmerkarl
ChrisGundry
Veteran
Posts: 259
Liked: 40 times
Joined: Aug 26, 2015 2:56 pm
Full Name: Chris Gundry
Contact:

Re: MSCS backups

Post by ChrisGundry »

Hi Karl,

Yes, Microsoft Cluster Services.

Are you saying you have those applications running on GUEST MSCS and are backing up with Veeam B&R?

The reason I raise the question is because the method to get GUEST MSCS working in VMware (and I believe the same is true for Hyper-V) is that you enable 'bus sharing'. Once bus sharing is enabled you cannot snapshot the VM in VMware, so Veeam cannot snapshot it, so you cannot back it up...

ps. I am very experienced with Veeam already, I have been using it for many years. It's just that I have not personally set it up to backup a MSCS scenario before. The research and testing I have done seems to indicate it is not possible.

Thanks

Chris
ChrisGundry
Veteran
Posts: 259
Liked: 40 times
Joined: Aug 26, 2015 2:56 pm
Full Name: Chris Gundry
Contact:

Re: Backup a MSCS cluster

Post by ChrisGundry »

Hi. Why has my thread been merged into this one with no explanation? This is a real problem with these forums...

I have already read this topic during my research. I asked again in a different topic because I am asking Veeam for their recommendations on backing up a MSCS setup using VEB or other method and asking users of MSCS to see what they are doing if VMware/Veeam do not allow this functionality/support VEB for MSCS etc.
widmerkarl
Expert
Posts: 122
Liked: 29 times
Joined: Jan 06, 2015 10:03 am
Full Name: Karl Widmer
Location: Switzerland
Contact:

Re: Backup a MSCS cluster

Post by widmerkarl » 1 person likes this post

ChrisGundryCEGA wrote:Hi. Why has my thread been merged into this one with no explanation? This is a real problem with these forums...
Yes, i was also a little irritated as i read this post, since i can't remember that there were more posts before my answer... Anyway.

I don't have any Microsoft cluster running at my customers, so i can't really tell you more about clustering.

What operating system is the cluster running? Server 2003 or newer? Since with Server 2008 and 2008 R2 the cluster functionality became renamed to "Windows Server Failover Clustering" and the Component Load Balancing (CLB) feature has been deprecated.

What virtualization platform are you using?

At least when i have a look to VMware KB i find this article: http://kb.vmware.com/selfservice/micros ... Id=1006392
Workaround

If you require bus-sharing:
Consider running backup software within your guest operating system as an alternative solution.
If your virtual machine currently has snapshots and it is preventing you from configuring bus-sharing, delete the snapshots. For more information, see the Using Snapshots section of the vSphere Basic Administration Guide for your version of VMware ESX.

Note: If snapshot consolidation fails, see Consolidating Snapshots(1007849).

If you do not require bus-sharing:

If your virtual machine is currently configured with bus-sharing, but you want to create snapshot, you must disable bus-sharing. For more information, see the Managing Mapped LUNs section of the ESX Configuration Guide for your version of VMware ESX.
For me it seems as you have the following possibilities:

- If bus-sharing is required, you have to use a backup software within the VM. For this i would recommend Veeam Endpoint Backup FREE, since you can backup in an existing Veeam repository.
- If bus-sharing is not required, you have to disable it.

Veeam create a snapshot of the VM (or tells vSphere to create a snapshot of a VM) and then retrieves the data out of this snapshot. Bus-sharing and snapshot don't work as VMware says.
Karl Widmer
IT System Engineer

vExpert 2017-2024
VMware VCP-DCV 2023 / VCA6-DCV / VCA5-DCV / VCA5-Cloud / VMUG Leader
Former Veeam Vanguard / VMCE v9 / VMTSP v9 / VMSP v9
Personal blog: https://www.driftar.ch
Twitter: @widmerkarl
ChrisGundry
Veteran
Posts: 259
Liked: 40 times
Joined: Aug 26, 2015 2:56 pm
Full Name: Chris Gundry
Contact:

Re: Backup a MSCS cluster

Post by ChrisGundry »

We are using VMware 5.5 U2 with Server 2008 R2/2012 R2 guest OS's

The issue is like I said, and like you have linked to is that to use MSCS/failover clustering you need bus sharing enabled on the VM's, which means snapshots cannot work with VMware/Veeam. If you have not used failover clustering then you won't have run into this issue luckily.

What I am asking is:
1. What other people are doing fro backups when using MSCS/failover clustering?
2. What backup Veeam recommend for MSCS setup? It's an enterprise MS feature and Veeam don't seem to have a function to support it unfortunately. This means we either need to use VEB (but only if its supported), use another vendors product, or not use MSCS...
3. What Veeam will/won't support in terms of VEB and MSCS. VEB is a free product which we do not have the paid support for like we do B&R, so what level of support could we expect from Veeam for VEB on MSCS.

Thanks

Chris
Vitaliy S.
VP, Product Management
Posts: 27375
Liked: 2799 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Backup a MSCS cluster

Post by Vitaliy S. »

Hi Chris,

1. To the best of my knowledge, some people are disabling bus sharing during backup job or reconfigure their infrastructures to be supported for VM snapshots.
2. As you correctly say snapshots for VMs with disks engaged in SCSI bus sharing are not supported by VMware, so it will not be possible to backup these VMs with any VM-image level backup solution.
3. You can try to use VEB to backup MSCS, but keep in mind that VEB doesn't bring any cluster-specific awareness logic in its backup jobs. Since VEB is not designed to backup MSCS, then will be no full support provided if something goes wrong during the restore process.

Hope this helps!
ChrisGundry
Veteran
Posts: 259
Liked: 40 times
Joined: Aug 26, 2015 2:56 pm
Full Name: Chris Gundry
Contact:

Re: Backup a MSCS cluster

Post by ChrisGundry »

Hi Vitaliy,

Thanks for replying... I don't really feel like I am getting anywhere at the moment.

1. Ok, but how exactly? As far as I can see you can't disable bus sharing during the backup because.
a. There is no mechanism to do that automatically that I know it
b. That will break the Microsoft clustering
That is why I am looking for responses from people that have actually done this with Veeam and why I didn't want my topic lumped in with an old topic which isn't the same thing...

2. So we are agreed that bus sharing prevents snapshots and Veeam from working at all.

3. Quite annoying then that when I asked the same question via a support case (01149371) I was told I should use VEB to backup MSCS, then was told I needed to raise a feature request, then told by another person in the same case that it was not supported. I raised some more questions and kept chasing for a reply but got no answer. I finally received a reply this week saying that the people on the ticket had got confused and apologise for the delay. I reply asking for the answers then and find that the ticket is closed! Its ridiculous, I was told I would have a reply on the 30th of November!

MSCS is a standard thing for any reasonable sized IT setup, so this should not be anything new to Veeam support so why is this such a problem?

Regards
NightBird
Expert
Posts: 245
Liked: 58 times
Joined: Apr 28, 2009 8:33 am
Location: Strasbourg, FRANCE
Contact:

Re: Backup a MSCS cluster

Post by NightBird » 1 person likes this post

I think because actually new generation of high available Microsoft applications like Exchange DAG setup, SQL Always on setup... don't need bus sharing anymore.
ChrisGundry
Veteran
Posts: 259
Liked: 40 times
Joined: Aug 26, 2015 2:56 pm
Full Name: Chris Gundry
Contact:

Re: Backup a MSCS cluster

Post by ChrisGundry »

That is true in part definitely and that is meaning that people don't use MSCS as much anymore, which is fine by me, I don't have much love for MSCS but it does provide a much needed HA service...

The issue to me is that a lot of people still do use MSCS and some of this technology like SQL always on is still new, so people don't have it yet. So thats not a reason for no one commenting with their working MSCS setup, or Veeam's inability to give me a straight answer about their products support of the various options... A KB article explaining that bus sharing is not supported by VMWare, so cannot be supported by Veeam and VEB isn't supported for MSCS would have saved a lot of hassle. I can't believe I am the only one who is using MSCS and also has Veeam...

We have Exchange DAG's here, which provide is with HA for Exchange, which is great.

The areas that we need to address within our network are:
SQL HA - We can't get our massive SQL 2016 costs approved at the moment so always on isn't going to happen. This does not stop us using MSCS to provide HA, but it seems Veeam can't back it up so we need to review options, hence the post.
File server HA - There are tons of options here but MSCS is one
Intranet sites - Again MSCS would prove handy for this
DHCP - We don't need MSCS for that anymore

I know there are other options to MSCS but training staff and managing MSCS for various purposes is a lot easier than doing the same for all of the other options that provide the same thing such as: DAG's, SQL always on, scale out file server, DHCP native failover, network load balancer for IIS etc

Does that make sense?
Post Reply

Who is online

Users browsing this forum: Bing [Bot], EskBackupGuy23, Ilya and 76 guests