QuObject on QNAP as an immutable repository (Backup copy failing)

[BostjanUNIJA]

Hi.
We have take over a customer with VBR environment where they have:
- primary backup repository (linux immutable)
- secondary (qnap with QuObject).

We are having a trouble with backup copies to QuObject QNAP.
We have recreated backup repository by following: "https://www.youtube.com/watch?v=84l0rYKFAow&t="
We have recreated backup copy job and ran active fully yesterday.
Today we have checked status of the job (and some VM's passed thru OK, some failed)
Errors are:

Code: Select all

22. 10. 2024 03:54:54 :: Error: HTTP exception: Failed to write request body, error code: 110
Failed to download disk 'SERVERNAME.vhdx'.
Reconnectable protocol device was closed.
Failed to upload disk 'SERVERNAME.vhdx'
Agent failed to process method {DataTransfer.SyncDisk}.

Code: Select all

22. 10. 2024 07:12:15 :: Error: Connection was forcibly closed.
Failed to upload disk 'SERVERNAME2.vhdx'
Agent failed to process method {DataTransfer.SyncDisk}.

What's goin on? Is this a network issue or this is a QNAP issue?

We haven't got the credentials for this VBR environment to open a support case yet, but we are working on that.

[david.domask]

Hi Bostjan,

Unfortunately we will not be able to troubleshoot this over the forums, so indeed a Support Case is the best option here. The errors posted look like network issues at first blush, but the cause is not clear and there could be other reasons the connection was forcibly closed.

Please open a Support case at your earliest convenience and share the case number here once created. Be sure to include logs for Support to review. (use the 1st radio option, select the affected jobs for export).

[DE&C]

For me this looks like your object storage system can not handle it (error response and from the description, and my experience from the field). This is expected with the solution you are using. Here you will find more background information why:
- object-storage-as-backup-target-f52/qna ... 39-30.html
- post489837.html#p489837

What you can try
"https://helpcenter.veeam.com/docs/backu ... ml?ver=120" ->set the concurrent task to 1

This is what I can say with this little information I have from your environment.

In general: DON'T use Qnap object storage for Veeam. You will have problems on the long run (see the link above from the other post for more information)

[BostjanUNIJA]

Thank you for your replies.
De&c - I have read the entire thread from URL that you have provided and now I understand that QNAP's QU Object solution just isn't ready for production environment.
I will probably go with SMB approach, since this QNAP is located on Site B.
I'm just thinking out loud (probably the answer is no) if I would create ISCI connection to volume on QNAP which is located on other site? Probably a bad idea, since its S2S connection between sites. Just go for SMB for backup copies, and maybe wasabi for offsite object storage.

[DE&C]

@Bostjan: You are welcome. Happy to hear that it helped

I wouldn't go with iSCSI if there is a high latency, VPN etc. between the sites - I would use iSCSI if everything is on the same site and the network is stable, has low latency and enough performance. ->"no" to your question.

If you need to do this and use a "file share protocol" I would suggest to use NFS instead of SMB as protocol (for the copies).

Input: If you have a VM or physical server in the same location as the QNAP, you could use this vm/server with iSCSI (no problems expected) or use it as gateway for the NFS share (https://helpcenter.veeam.com/docs/backu ... ml?ver=120).

A cloud provider with immutable object storage is for sure a better idea for the offside copy (you can solve multiple challenges at once

[haslund]

@Bostjan: Did you create a support case for this issue? If yes, could you share the case #, please?

[BostjanUNIJA]

Hi.
Thank you for your reply.
No, we haven't opened a support ticket.
We have resolved the issue:
- Suggested customer to have a 2nd backup copy to cloud object storage (Wasabi) - customer said no.
- We have "ditch" QObject on Q-Nap and now backup copy over Site2Site vpn over SMB network share.

I know that they are loosing bunch of functionalites with that solution (like fast cloning, etc), but they just wouldn't want to get rid of Qnap or copy their backups to cloud.

B

[Gostev]

We highly recommend against non-HA SMB shares on low-end NAS as backup target from reliability perspective (based on nearly two decades data loss statistics in our support). Especially over remote links, which will multiply the likelihood of data loss by an order of magnitude.

Be sure to relay them this information to the customer so it's not your fault when they are unable to recover and hear the same from our Customer Support.

This is even before the fact that they still don't have a single immutable backup so hackers will be able to delete all of their backups at once.

[BostjanUNIJA]

Gostev thank you for your reply.
We will act as you suggest.
I know that SMB over Site 2 Site vpn tunnel is bad.
At least they have 2 backup repositories for now.
1 local linux hardened repository
1 backup copy to SMB share over Site2Site.
We will suggest air gapped / tapes in the near future.
Have a nice day!
B