Under certain failure conditions, all replica restore points will be removed, and the state of the replica disk recalculated. The resulting digest file is fairly small, and is stored in the repository for replica metadata (which should be located on the source side, or on the source proxy server itself). Removing the restore point snapshots can in some cases be time consuming, but the actual digest calculation requires that the target-side proxy read the entire replica disk. The bottleneck for this process is usually either the location of the proxy (if for some reason a proxy across a slow link from the replica datastore was accidentally selected), the transport mode (whether network or hot-add is faster is somewhat variable, but hot-add tends to win), or the speed of the datastore disks. All things being equal, you could expect a digests calculation to take as long as an active full backup of your production VM to an arbitrarily-fast repository, but if your replica storage is slower than your production storage that could be the problem.
Ideally, the job would not be failing at all; you might consider opening a support ticket to have the failures reviewed.