REFS Write amplification?

[fillerbunnie]

Hi all,

We're looking to replace our existing backup tools and thought we would give Veeam a shot as we support a lot of customers who use it quite successfully (with Hyper-V), however we're seeing some very unusual? performance figures.

We run all-flash Windows REFS backup repos, which have no issues sustaining 800+MB/sec sustained write from 10+ write streams using our existing backup product. Testing vmware backup of a cluster, Veeam reports 250MB/sec processing, with 99% bottleneck being the target, yet the target is actually writing backup data to the REFS volume at 500-800MB/sec (based on process monitor on the repo, 500-800MB/sec of Veeam processes writing to the VBK files). The total data written to disk equates to about 250MB/sec as well (nowhere near as big as would be expected if it was actually writing 500-800MB/sec to disk).

It's like Veeam is doing some extreme write amplification - I can't imagine this is a bug as it was a clean Veeam install and a newly formatted REFS 64K volume, no deduplication or anything fancy. Veeam we used all the default settings, normal incremental, optimal compression and all the defaults for the repository. This was an initial backup (so I guess a full read of the source VM's), but the same thing was observed when I re-ran the job as an incremental backup.

Performance aside, the 2-3x write load on the flash repository alone would be enough for us to reconsider the product.

Is this normal/to be expected from Veeam, or if not has anyone else seen this kind of write amplification with REFS?

Cheers!

[HannesK]

Hello,
and welcome to the forums.

If you say the speed is lower than with another product and 99% bottleneck "target", I have one guess without knowing more about your infrastructure: https://helpcenter.veeam.com/docs/backu ... ml?ver=110 - per-machine backup files is unchecked (default setting). On the other hand, you say "VBK files". So my guess might be wrong.

The write load to the backup storage should be around half of the read load from the production storage. REFS is not that special, that there would be massive difference to notice. Example: you back up from source with about 1GByte/s, then you should see about 500MByte/s write speed.

REFS volume at 500-800MB/sec (based on process monitor on the repo, 500-800MB/sec of Veeam processes writing to the VBK files)

The total data written to disk equates to about 250MB/sec as well

I assume that one of the values you mention is wrong. Support can help you to check it (please post the support case number). It's hard to guess via a forum.

Best regards,
Hannes