Proxy... Proxy... But what IS IT ?

[AlexLaforge]

Hi,

OK : I'm stupid and I have to leave IT, and start selling Ice-creams on the beach. Thats would probably be better for me.

After having bought Veaam BR 9.5, having spent 2 weeks reading tutorials, extensively read the Veeam B&R Documentation, I'm still at the same point : I do NOT know what Veeam mean by "a Proxy".

On this page https://www.veeam.com/blog/vmware-backu ... ation.html, and in the Documentaiton of Veeam B&R they are always referring to the "Proxy" concept. But I'm tired of trying to understand by myself what they mean by this "proxy" concept.

I'm about to give up.

I have a very basic need. The simplest need in the world. The simplest infrastructure. Not a Fortune-500 company with a 50 people IT-team full of Knowledge and paid 50 K per year. Veeam should really take 5 minutes to add clear explanations of what is a "Proxy" for the Veeam excellent IT guys.

My infrastrucutre ? The most basic :
- Install Veeam B&R in a VM on the ESXi #2 (DR Site).
- Veeam B&R will connect to ESXi #1 (Production Site) everyday to take backups of 1 VM maybe 2 (wow !)
- When the ESXi #1 will fail, I will restore the VM backups on the DR site (ESXi #2), or use "Instant Recovery".

So : What is a "Proxy" ? Is a "Proxy" a "Veeam Installation" ?
This page of the Docs say that "For offsite migration, you must deploy at least one backup proxy in each site" :
https://helpcenter.veeam.com/docs/backu ... tml?ver=95

- Do I have to install Veeam B&R in a VM on my DR Site (ESXi #2), AND on the Primary Production site (ESXi #1) ? WHy would I do such a thing ?
That would be useless : My DR ESXi #2 will backup the VM from ESXi #1, not from a useless Veeam VM on ESXi #1
How would these two VM's communicate ?

Moreover, the license allow to use the Veeam B&R on one machine at at time... That's a terrible nonsense.
Or... maybe this is not clearly explained in your docs ? Or maybe I'm blind. Or maybe upset in the end. Probably.
What IS a "Proxy" in the Veeam point-of-view ? What does this mean for their team ? Am I so stupid that I have to let down my 15-years IT knowledge ?
Probably.

But in the eventual case someone understand my psychological state-of-mind, I would appreciate some lights on this topic. Becasue I'm sure that when I will have got what this mean, the Doc will become luminous and I will be able to finally set-up my infrastructure !

And Yes : I thank you very much if you can give me 2 cents of help

[DaveWatkins]

So, lets cover some basic things, Veeam has "roles" these roles can all live on one VM or server, or be spread around to increase backup speeds to where it makes sense.

For example if you have a tape drive connected to a physical server via SAS, then you would install the Tape role on that server. Your actual B&R server may still happily run in a VM, but installing the Tape role on the physical server with the tape drive attached means you can backup to that tape drive.

Other roles include the "repository", the "proxy", the "WAN Accelerator" etc.

The proxy role contains the Data Mover service which does the compression and dedup as the data flows through it. In ESX what it actually does is attach a snapshot of the VM being backed up disks directly to the proxy and then simple copies the data off it. This means load on the VM being backed up is basically nil.

In your case you'll want a proxy server in your primary site, and the proxy service also running on your B&R server and you should be good to go.

You can have as many roles installed on as many servers as you like. The license is only restricted by the amount of CPU's you have licensed. You can even have multiple B&R servers running (in different geographic areas for example) as long as your total CPU's consumed are less than or equal to number you have licensed.

Finally, to add a proxy add the server under Backup Infrastructure in the console.

[AlexLaforge]

Hi DaveWatkins,

First, let me thank you : Thank you !
I usually keep calm, but IT can drive crazy. IT got me this time.

So a "Role" means both : "A Resource used by processes", and "A Process using resources" ? That sounds strange, but OK, let's assume that.

For what I understand :

- I will use 4 "roles" :
- The "Backup" role (a process). This role will be installed on my VM running Veeam.
- The "Repository" role (a resource). This role will be on a local hard drive on my VM running Veeam.
- The "Proxy service" #1 role (a process). This role will allow Veeam to connect my ESXi #1 (a resource) and backup the snapshots from it.
- The "Proxy service" #2 role (a process). This role will allow Veeam to connect my ESXi #2 (a resource) and restore a backup on it, or run "Instant Recovery" on it.

So, when the documentation say that "For offsite migration, you must deploy at least one backup proxy in each site", and when you are talking of "proxy SERVER in your primary site" and "proxy service running on my Veeam VM", I understand that in fact it means that I only have to add my ESXi #1 and my ESXi #2 servers inside Veeam. There is nothing to install as "proxy SERVER" on my ESXi #1 ? I understand that my ESXi #1 IS a proxy server, my ESXi #2 IS a proxy server, without the need to install anything else on it ?

- With that setup I described above, Veeam will be able to connect to the ESXi #1 and gather backups/snapshots, and Veeam will be able to connect to the ESXi #2 to deploy/restore my backups ? Am I right ?

Did I finally got it ?

[DaveWatkins]

Not quite.

The proxy role is literally a service, specifically a service that run on a windows machine (as a windows service even).

You can add the role to any windows server, obviously the B&R server on your second site makes sense there, you can either dedicate a VM to it on your other site or add it to an existing VM. There are some restrictions about how the VM with the proxy service is backed up but at least initially it's probably not worth worrying about except to say you're better off putting the proxy service on a VM that doesn't have a lot of change unless you're going to dedicate a VM to it

[AlexLaforge]

Hi Dave,

Thank you.. that's getting a little bit clearer..but not totally ! (I am sorry about my "reluctant" mind on this topic).

Just to clarify :
- My "Production site" is on my ESXi #1:
- My "Backup site" and My "Recovery site" are on the same ESXi #2:
- My Backup site is the ESXi #2 on which I have only 1 VM installed : The VM dedicated to Veeam B&R. I t will backup my ESXi #1.
- This ESXi #2 "Backup site" will be used as my "Recovery site" in case of disaster
They are, in fact, on the same site.


As I will dedicate only 1 VM to Veeam B&R, based on what you say, I assume that this VM will have ONLY 1 PROXY SERVICE (running as a Windows service), so I will have only 1 VM with Veeam dedicated to this "backup/proxy/repository" role... Am I right ?

But I still do not understand why the docs are evocating "several proxy servers" ?.. Literally, I do not see how could I have more than "1 proxy" on the same Veeam B&R VM ?
.. Can I have multiple "proxies" on the same VM ?

NOTES:
I will not install Veeam B&R on my production ESXi #1.
Veeam will only be installed in my ESXi #2 VM.

Thank you for the time you are taking, thus helping me to make this topic more clear in my blurry mind ^^

[tsightler]

I'm going to try to add some explanation here, I'm not sure it will help, but I'll give it a shot.

If you think about what the term proxy means, it is anything that has been authorized to take action on something else's behalf based on their request. In the case of Veeam, it is a service that can be asked to perform work on behalf of the Veeam server.

So yes, you may have a single VM, with Veeam installed, and it will have a proxy service installed locally. Technically, this single VM is all that is required for your setup to function, however, it might not be the most efficient way. I assume your two sites are connected by some type of WAN, if so, this WAN will likely have limited bandwidth. Certainly the Veeam server installed at the DR site, or more specifically it's proxy component, can read the data from the Production site, but when it does so, it would be the raw data, with no compression or dedupe. The DR site Veeam server/proxy would not have direct access to VMs/storage at the production site, so it would have to retrieve the data via the ESXi host, which would send that data across the network to the DR site prior to Veeam being able to process it.

On the other hand, if you have the Veeam server deployed at the DR site/ESXi #2, and you also deploy a small VM at the Production site on ESXi #1 you can add this small VM from ESXi #1 as a proxy server to the Veeam server at the DR site on ESXi #2. The Veeam server would now have it's local proxy in the DR site (on the same server Veeam was installed), and a remote proxy in the Production site. During a backup, Veeam could contact this Production site proxy and have it read the data locally on it's behalf, this remote proxy could then compress/dedupe the data prior to sending it across the network to the Veeam server at the DR site, saving bandwidth and time.

That's really all a proxy is, another computer/VM that has been added to Veeam as a manager server, and then enabled for the proxy role, which tells the Veeam server that it can use these other computers to do work if it determines that it is more efficient for them to do so. It is not "required" to have additional proxy, but it can be more efficient.

I would suggest the following reading and instructional videos:
https://helpcenter.veeam.com/docs/backu ... tml?ver=95
https://bp.veeam.expert/resource_planni ... modes.html

And the first modules on Veeam University do a pretty good job going through the architecture components, including proxy servers:
https://www.veeam.com/university.html

[AlexLaforge]

Hi tsightler,

You shot right ! This "proxies things" are now a lot clearer ! I must thank you for having taken the time to explain it so clearly. And moreover, thank you for providing these useful links, especially the https://bp.veeam.expert : No post thread, no official Veeam doc, no Veeam white papers, and no toturial NEVER sent me to this gold mine !

So, Il will try to resume, and hope that you will be able to "validate" my assumptions :

1. From what I had read in the docs, I was thinking that Veeam B&R installed on my VM ESXi #2 (Backup server) was requesting the production VM ESXi #1 to perform compression and deduplication on the ESXi #1 before transferring to ESXi #2 (Backup server).
   Your answer now permits me to understand that THIS IS NOT the case, and that if I want compression and deduplication to occur before the transfer, I need a backup proxy located on my source ESXi #1 production server. So I will go this way !
   I have read that the deployment of this backup proxy agent is fully automated — "Veeam Backup & Replication installs the necessary components on a Windows-based server when you add it to the product console. The backup proxy can be a VM with HotAdd access to VM disks on the datastore. This type of proxy also enables LAN-free data transfer.
2. Do I need to set-up, on my ESXi #1 (Production site), a small dedicated Windows VM that will act as proxy ? Or Can I use one of my backuped VMs acting as a proxy (I think that would cause problems — a Vm proxying its self-backup...mmm..not so good—, so I would go for a dedicated Windows VM as proxy on my ESXi #1 Production site). What do you think about it ?
3. Does this Windows VM acting as proxy need a lot of disk space ? Or can I keep it very small ? Does the answer to this question depends on the transport mode I will use ("Hot Add" or "Direct Storage access") ?
4. The docs states that "During deployment, it is determined which datastores the proxy can access." : So When the Veeam "Proxy Server" on ESXi #2 (Backup site) will be commanded to use the "Backup Proxy VM" located on my ESXi #1 (Production site), Veeam will automatically know that this Proxy backup VM has access to the "production datastore" that resides on the same ESXi #1 than the backup proxy, and then Veeam will use this "production datastore" as temp local storage, thus speeding up the process ? And after having compressed the backup on ESXi #1 (Production site), Veeam will just transfer it over WAN to the ESXi #2 (Backup site) ? Did I shot right ? (I hoooope so!)
5. As the target backup storage server is ESXi #2, it is not located on my production Server ESXi #1. So I believe that I will not be using Veeam in "Virtual Appliance", and I will have to choose "Direct storage access" rather than "HotAdd" ? Right ? False ? It depends (Lol) ?

[cvlr]

1. yes - if you want compression and dedupe before transfert you need proxy at the source side
2. it does not need to be a dedicated machine. It would be preferred though.
3. it is preferred to keep it as small as possible, because if you use hot add mode, veeam will disable vmware cbt on that vm. So small = good.
4. veeam itself does not use space on the datastore. veeam instructs vmware to take a snapshot of a vm that needs backup. then veeam attaches the vm disk to the proxy server and reads it to compress/dedupe and then end ot over to the proxy on the esxi #2
5.if ESXi #2 does not have access to the same storage of ESXi #1 you will need to use hott-add

[tsightler]

Yes, it is highly preferred to have a dedicated machine for a proxy, but it is not an absolute requirement. If you have a small Windows machine that meets the requirements then it is possible to use it, however, as you noted, if you think about the aspects of a proxy VM backing itself up, well, that's perhaps not great but is at least possible, however, it's totally impossible for a machine that is a proxy to restore itself. There are also a few other limitations for a proxy VM backing itself up. For all of these reasons it's highly recommended to use dedicated VM for a proxy. Note that if pricing is a concern the proxy VM does not have to be a server OS, you can using Windows 7/8/10 just fine.

Just a small point of clarification regarding storage on a proxy, the proxy does not store any data, the compression and dedupe is performed on-the-fly. In your case the proxy on ESXi #1 would hotadd the snapshots of the VMs on ESXi #1 then read and stream the data across to the Veeam server on ESXi #2. None of the data is actually written to the proxy, the proxy is simply the conduit through which the data passes, and provides processing power to compress and dedupe the data before sending it across the network.

BTW, the https://bp.veeam.expert/ is actually referenced on the main Veeam documentation website, https://helpcenter.veeam.com. It shoudl be the second link in the Product Guides section of this page.

[AlexLaforge]

Thanks to @clvr and @tsightler, I think I'm getting close to a fully understanding.

1. I will use a proxy at the source side (mandatory).
2. I will prefer using a dedicated VM for a proxy, definitely.
3. I prefer using a small VM in order to have it take not too much space.
4. Very instructive to know that it works this way ! Thank you
5. From what I have read, this "HotAdd" method is only possible if the backup repository and the backuped source VMs are on the same SAN. That is not my case, so I think taht "HotAdd" won't be usable. This "HotAdd" concept remains my last problem. So I will rely on the following solution from the Veeam documentation : If a datastore cannot be accessed by Direct SAN or Virtual Appliance (HotAdd) mode by any proxy, then the system will use Network mode to retrieve the VM disks via the ESX(i) management interface.

Thank you a lot !

[tsightler]

5. From what I have read, this "HotAdd" method is only possible if the backup repository and the backuped source VMs are on the same SAN. That is not my case, so I think taht "HotAdd" won't be usable. This "HotAdd" concept remains my last problem. So I will rely on the following solution from the Veeam documentation : If a datastore cannot be accessed by Direct SAN or Virtual Appliance (HotAdd) mode by any proxy, then the system will use Network mode to retrieve the VM disks via the ESX(i) management interface.

This part is still not correct. This backup repository is where your backup data is stored, which in your case is obviously going to be at your DR Site/ESXi #2. It is the proxy that must on the same SAN as source VMs, which is exactly why it is preferred to have the proxy installed on your Prod Site/ESXi #1 server, so that it can perform the hotadd function, compress and dedupe the data, and stream it across the network to the backup repository on your DR Site/ESXi #2 site. The proxy at Prod Site/ESXi #1 is doing work on behalf of the Veeam server because it on the same SAN as your source VMs. If you do not configure a proxy on the Prod Site/ESXi #1 server then the proxy on the DR Site/ESXi #2 server would have to use Network mode to pull the data since it is not on the same SAN.

That's one of the primary reasons that proxies exist, so that you can place them "close to the source data" regardless of where the rest of the components live. The other is of course so that you can scale out resources for larger environments.

[AlexLaforge]

OK @tsightler, So I will need to use "HotAdd" storage mode on my proxy VM located on my "Prod Site/ESXi #1".

That (finally) sounds logical to my mind now !

Sorry for the annoyance, and I really appreciate the time you have taken in explaining better this "proxying things"
I hope it will help more people in the future

[AlexLaforge]

Hi,

Simple question about the "backup proxies" :

I have two ESXi Sites :
- Production ESXi #1
- Backup AND Recovery ESXi #2

My Disaster Recovery ESXi #2 is also my Backup site. It actually contains only 1 VM.
This VM have Veeam B&R installed on it, and Veeam B&R stores the backups on a NTFS volume that is part of this VM.
(These backups are retreived from my Production ESXi #1 through a backup proxy located on my Production ESXi #1)

When I will need to restore a VM from a backup (located on ESXi #2) on this Disaster Recovery ESXi #2, I assume that I will not need another "backup proxy", as the main installation of Veeam B&R is already a "backup proxy" ?

[Gostev]

Yes, unless you've manually removed the default VMware Backup Proxy that is created automatically when you install the product from your backup server. Some people do that, because they prefer backup server to only perform job management tasks - to keep its CPU from being overloaded by data processing.

[AlexLaforge]

Oh GOD : I finally assumed something GOOD for the first time by Myself ! Great
I will not remove the default VMware Backup Proxy that is created automatically when I install the product : So .. That's perfect .
Thanks !

[mikegodwin]

I understand that with VMware vSAN, the ideal case is to have one proxy located on each physical ESXi host so reading data is handled locally on each host. This reduces network traffic, but my question is, does this increase write throughput to the backup repository? I have more LAN bandwidth than I will consume (25Gb switches) so reducing some network traffic is not that important, but if I can reduce backup times by placing an on-host proxy on every ESXi host, then I will do that.

[DGrinev]

Hi Mike,

The main purposes of the backup proxy are to compress, dedupe and move backup data in a most optimal way to the target repository.
A physical proxy recommended only in case you are going to utilize Direct SAN transport mode.
In order to reduce backup time you should review bottleneck stats of the job and improve founded weak spot through bottleneck analysis.

Please review this existing thread for additional information. Thanks!

[mikegodwin]

Thank you for the reply. I believe VMware vSAN does not support Direct SAN transport mode, is that correct? From what I read it is advised to use appliance mode/hotadd, but to still have an on-host proxy on every ESXi host.

https://www.veeam.com/kb2273

Maybe in the end I will just need to try it with a proxy on each host and see if it improves performance significantly.

[DGrinev]

I believe VMware vSAN does not support Direct SAN transport mode, is that correct?

That's correct.

Deploying proxies on each host is highly recommended, as it should allow achieving better performance and reducing the backup window. Thanks!