ReFS repository slower than NTFS repository

[jeffshead]

------------------------------------------------------
Veeam B&R 11
Veeam MS Agent 5
vSphere 7.0u1d (soon to be updated to U2)
------------------------------------------------------

ReFS seems to be quite a bit slower than NTFS. At least when creating the first, full backup and incrementals.

I have a VM with a 16TB, independent disk. Since I need to back up the entire VM and run pre/post job scripts, I created a B&R managed agent job for it. Both the VM being backed up and the VM on which B&R is installed are on the same ESXi host. The destination for this backup job is an external USB drive (repository) attached to the B&R server VM via PCI passthrough.

I originally formatted the drive with NTFS and ran a few full and incremental backups. Full backups consistently took 4.25 hours and incrementals took 5-6 minutes. As a side note, this was really good compared to my previous set up that was on different hardware. Those same full backups took 8-9 hours.

With this new job, I have it set to create full synthetics. This took longer than creating the initial full backup so I figured I'd try the same setup with ReFS. I deleted everything and reformatted the USB drive with ReFS (64KB cluster). I then recreated the repository and backup job with the same settings. The new full backups take 6 hours and incrementals take 8 minutes. No data or file changes on the VM being backed up.

Is this typical when comparing ReFS with NTFS repositories? I ran a few disk bencmark softwares and they all seem to indicate NTFS read/writes are much better than ReFS. I haven't created a synthetic full on the ReFS repository yet. From a bird's eye view, I'm hoping that will make up for the slower full backup writes. In this particular use case (USB drive, limited space) wouldn't it make more since to just go back to NTFS Forever Forward Incremental Backups? What would be the advantage of sticking with ReFS?

[Mildur]

I would never use refs on a rotated disk or if you configure only active fulls.
From a veeam perspective, FAST Clone can not be used sometimes, because with rotated disks, your old full backup and data are not available, if you plug in a new usb disk.

With rotated usb disk, you can stay on ntfs and will not loose any veeam optimization.

If you have non-rotated usb storage, you will have a very fast synthetic full, faster as ntfs synthetic full. Because Veeam can use FastClone to create the new Full backup file. Fast Clone will be faster as your „active Full test“, because not every block from the 16TB will be saved again on the usb disk. Veeam can reuse most of the blocks already saved on the usb disk to synthesize a new full backup. On ntfs, the block needs to be copied. On refs, the block will only be a reference in the new file.

https://helpcenter.veeam.com/docs/backu ... ml?ver=110

[jeffshead]

Thanks for the reply. I'm currently not using rotated disks nor active fulls.

I don't know that I have a need for periodic fulls (active or synthesized) so in my use case, I don't see the advantage of sticking with ReFS with synthesized fulls over using NTFS with Forever Forward Incremental Backups (no periodic fulls).

From a reliability (less chance of corrupted backups) point of view, should ReFS with synthesized fulls be considered a safer choice to NTFS with Forever Forward Incremental Backups (no fulls)? I don't see how since synthesized backups will contain the same blocks as the incremental backups.

[Mildur]

From a reliability (less chance of corrupted backups) point of view, should ReFS with synthesized fulls be considered a safer choice to NTFS with Forever Forward Incremental Backups (no fulls)? I don't see how since synthesized backups will contain the same blocks as the incremental backups.

I would say, both are the same level if you look at possible „data corruption“.

Forever Forward Incremental needs a Full too.
And this full will use old incremental blocks to transform the chain.. The oldest incremental will be combined with the oldest full to create a new Full Backup each day.
https://helpcenter.veeam.com/docs/backu ... ml?ver=110

The transformation process is performed in the following way:

1. Veeam Backup & Replication adds a new incremental backup file to the backup chain and detects that the number of allowed restore points or days is exceeded.

2.Veeam Backup & Replication reuses empty data blocks in the full backup file to include changes of the incremental backup file that follows the full backup. To do that, Veeam Backup & Replication injects data blocks from the first incremental backup file in the chain into the full backup file. As a result, the full backup file ‘moves’ one step forward in the backup chain.

With Forever Forward Incremental, all files in the chain needs to be in a healthy state. If one vib file is corrupted, all backups after this vib file are useless.

Periodic Active full is the only backup method you can use, which doesn‘t use the old blocks from other restore Points. Fresh backup files without needing old blocks

[tminkov]

I have a big expereince with ReFS, will answer about its performance. As by design ReFS is transactional FileSystem - data is first written then marked as written. This makes it a very reliable file system. End with theory.
On practice by default the transactional of ReFS is only metadata, not data. This means that filesystem as metadata/structure is very reliable, but data maybe lost/corrupt.
If you want to be fully transactional - both metadata and data, you should use CLI format command with respective option /state:enabled. It is by default disabled for regular hard disk and "enabled for storage that supports it".
Again on practice, if both metadata and data transactional are enabled, the filesystem is remarkable slower.
As example: deletion of plenty of files (huge data and metadata) is very slower and after the end of this the actual freed space is released within 2-5 minutes.
My experience with Veeam is just in beggining but as I found it works fine with ReFS as Repository.
That what i found is if ReFS is used as backup source it dont support CBT native but Veeam CBT driver should be installed.