Assuming this problem happens only with nworks Collector servers - then I would investigate the Monitoring Load on each Collector. If they are monitoring too many Hosts + VMs, then the Ops Mgr Agent Health service can use high cpu and memory, which could delay heartbeats. the Agent might even auto-restart, if the correct configuration is not applied.
There are 2 key points to configure the agent for stable performance -
1. You must run the Configure Ops Mgr Agent Task against every nworks Collector (this task is built-in to our MP)
2. You must override a couple of Ops Mgr default monitors for PrivateBytes usage - this prevents the auto-restart.
Please see our Install Guide, page 13 for instructions on configuring the agent as above.
Hope that helps!
Both of the above