Architecture Assistance

[BHB3805]

Overview of the Environment
Currently using Veeam 7 for backup and replication of our environment.
We have two sites, the Primary DC (Production and DMZ) and Secondary DC (Disaster Recovery).
The sites are connected via 30 Mbps MPLS for enterprise data and a dedicated 1 Gbps Ethernet link for replication traffic.
The Production environment has a 3 host ESXi 5.5 cluster supporting 81 VMs and 12.5TB of data.
• The hosts are attached to storage (EMC VNX5300 – Block) using fiber channel.
• VMs backed up to drive on a physical Veeam proxy server.
The DMZ environment has a 2 host ESXi 5.5 cluster supporting 11 VMs and 500GB of data.
• The hosts are attached to storage (EMC NS-120) using iSCSI.
• VMs backed up to a CIFS share defined on a physical Veeam proxy server.
The DR environment has a 3 host ESXi 5.5 cluster, hosts are connected to storage (CX4-120) using iSCSI.
Backup Architecture
The production backup architecture consists of:
• Veeam Backup Server – virtual machine. Veeam application and database. Dual network interfaces connected to the enterprise and replication networks
• Veeam Proxy / Repository (Production) – physical machine. Fiber Channel HBA and dual network interfaces. FC connection to allow san transport for backup jobs. Network interfaces connected to the enterprise and iSCSI networks (Connection to iSCSI backup target).
• Veeam Proxy (DMZ) – virtual machine. Connected to enterprise network.
• 3 Veeam Proxy Servers (REPLICATION) – virtual machines. Dual network interfaces connected to the enterprise and replication networks. Used for replica jobs.
The DR backup architecture consists of:
• 3 Veeam Proxy Servers (REPLICATION) – virtual machines. Dual network interfaces connected to the enterprise and replication networks. Used for replica jobs.
Backup Operations
Backups are performed on a daily basis, synthetic full backups occur on Friday evenings. 30 restore points are maintained. We are experiencing the following issues:
• Daily backup jobs processing rate is lower than expected – 70 – 95MB/s – for some jobs. It is not consistent for the VM being backed up – varies.
• Large synthetic fulls (700GB – 1.5TB) take a long time to complete – 18 – 60+ hours.
Replication Operations
Replication jobs are performed on Monday through Saturday. Jobs are permitted from 4:00AM to 8:00PM. Monday – Friday and 2:00PM to 8:00PM Saturday. The jobs are configured to run every 2 hours, the goal is to run at 1 hour intervals. We are experiencing the following issues:
• Servers that have a suspected large change rate, i.e. Exchange, SQL Server, etc. take a very long time to replicate; exceeding time windows, generating orphaned snapshots, etc.
• In our current environment, we can’t replicate VMs on a consistent, timely basis.
Goals
Our goal is to determine what will be necessary, financially and operationally, to allow us to perform the following:
Best Practices
What changes should be made to adhere to Veeam best practices?
Are we missing components? Should we have an additional backup Server in the Seconday Data Center?
Backup Operations
• Increase the number of daily restore points to 60 (days). Implement 12 monthly restore points.
• Decrease the backup window. Determine what the bottleneck(s) are and address them.
Replication Operations
• Replicate all VMs on a 1 hour interval the Secondary DC on a consistent basis.

[Shestakov]

Hello Dan,

What changes should be made to adhere to Veeam best practices?

Among Veeam best practices is a concordance to 3-2-1 rule, I didn`t find among your description if you keep backups on physical medias or in a cloud.

Are we missing components? Should we have an additional backup Server in the Seconday Data Center?

Do you have WAN acceleration? It can help you fasten backup copy and replication jobs.
Single backup server is enough for your environment.

• Increase the number of daily restore points to 60 (days). Implement 12 monthly restore points.

You can use GFS retention policy capabilities for backup copy jobs to keep monthly backups with their own retention policy.

• Decrease the backup window. Determine what the bottleneck(s) are and address them.

Could you specify the issue with the backup window?
To determine bottlenecks, you need to review your jobs` stats. Check it we will think if it possible to fasten them.

And the last, but not the least:

Currently using Veeam 7 for backup and replication of our environment

On this week we released a new version of VBR with a HUGE number of new capabilities. Please take a look.