Data Compression & Deduplication - Reducing Offsite Job's Network Usage

[ColterT]

Not an error or technical issue - just confirming some troubleshooting conceptual ideas/understandings!

Good day! I am researching means of analyzing and reducing data sent from our client's server to our CC datacenter due to overage charges they're seeing on their monthly ISP bill. These occurred only in the last couple months, so I do not believe it's necessarily Veeam.

This customer's server is at their home and they do have WiFi, so I am looking into that as a probable cause of this overage as well.
We have a Veeam Cloud Connect server that we are able to confirm maintains 508GB backup chain - and over two weeks there's been < 100GB of data in incrementals, according to the cumulative incremental file sizes - but I'm wondering about compression. Could someone please do a sanity check on my assumptions:
1. Compression happens on the server-side, decompression happens on the offsite repository.
2. Therefore, incremental sizes are a reasonable approximation of total Veeam-sent-data over the customer's network.
3. Since the client has Forever-forward incremental backups, I can expect the transform/merge of the chain to occur on the offsite server.
4. Which means the merge isn't being done on the client's server, and the huge new .vbk is not being sent over the network.
5. If the above assumptions are true, it stands to reason the data is being chewed up elsewhere and I need to find a means of analyzing this customer's network traffic - and Veeam is not the cause. Right?

[chris.childerhose]

What is the job set up for in the Advanced Storage settings for Compression and Storage Optimization? You might want to consider the Storage Optimization changing to WAN target to see if that helps. Check this here - https://helpcenter.veeam.com/docs/backu ... ge_vm.html

[Mildur]

Hi Colter

Is this about Veeam Agent Backup Jobs directly to a Veeam Cloud Connect provider?

1. Compression happens on the server-side, decompression happens on the offsite repository.

The Agent does the compression before sending the backup blocks over WAN.

2. Therefore, incremental sizes are a reasonable approximation of total Veeam-sent-data over the customer's network.

Correct. There should be some overhead of course. Management Traffic. But that shouldn't be to much.

3. Since the client has Forever-forward incremental backups, I can expect the transform/merge of the chain to occur on the offsite server

Yes, merges happens on the Cloud Connect Repository.

4. Which means the merge isn't being done on the client's server, and the huge new .vbk is not being sent over the network.

Correct, only the incremental backup data will be send over WAN. The Repository does the merging for the Agent.

5. If the above assumptions are true, it stands to reason the data is being chewed up elsewhere and I need to find a means of analyzing this customer's network traffic - and Veeam is not the cause.

How much is the difference between Veeams incremental size and what you see on the firewall/bill? If you have a good firewall, you could use it to analyze the traffic send over the different ports/services.

Thanks
Fabian

[ColterT]

Fabian, thanks for the reply! This is client's VBR server with backup copies sent to a Veeam Cloud Connect provider (my company's servers).
The difference between Veeam's incremental sizes (totalling ~120GB over a month) and what I see on the bill is 900GB-1TB of difference. I knew it wasn't entirely Veeam's traffic but I wasn't sure if I could make the assumptions before confirming with you guys. Now I know Veeam only accounts for ~10% of the traffic and we need to look elsewhere. Too bad their equipment doesn't allow for decent traffic analysis.

Thanks again!