BTRFS or OpenDedup - Does someone use this in production?

[mkretzer]

Hello,

does someone use BTRFS or OpenDedup for deduplication of the backup repo in production? Any experiences?

Markus

[NTmatter]

I've had a bit of a play with BTRFS Deduplication (as of mid-2015) for storing long-term archives. It was technically functional, however the userland utilities were rather lacking and occasionally crashy. Data was properly deduplicated, but the tools had to rescan the entire dataset with every run (problematic for a growing dataset), with caching and such on the "real soon now" list. The utility also crashed on some larger runs, at which point I gave up and switched to periodic NTFS Deduplication, which I'm currently using for a small set of very long-term archival backups. I'm currently seeing 92% deduplication rate after six months of weekly fulls. This unfortunately means no native bitrot detection/recovery, but Veeam has its own checksums for that purpose.

As some other highly anecdotal experience, I recently tried using BTRFS as a Ceph OSD Backend. The snapshot create/delete workload trashed several disks to the point that they would lock up a machine when the BTRFS repair tools kicked in. I'll grant that the storage node was woefully under-provisioned, however low memory conditions and a hard lockup shouldn't render a filesystem unusable. I had to stick the disks into a non-Linux machine just to avoid the automatic fsck so I could zero out the drives.