Without deleting retention points our merges doubled in speed! We are not at the original "fast clone" speed from the beginning but its much better. Even an active full does not immediately stall when a merge is running in parallel.
Right now we have enough space to host even another week so we will see if the merges will even get faster next weekend.
I wonder: is there anything i can optimize with better hardware for the repo server? I simply cannot see which ressource is limiting the REFS to merge faster/delete restore points without issue. RAM is no issue anymore since we increased to 384 GB, CPU usage is low and disk queue of the REPO is 0. Buying a new server would still be cheaper than buying more storage and going back to NTFS