-
- Lurker
- Posts: 1
- Liked: never
- Joined: Aug 07, 2012 7:33 pm
- Full Name: Mark Muyskens
- Contact:
Unable to retrieve next block transmission command
Hello,
I am trying to setup Veeam to use an Amazon EC2 instance as a backup repository and I am running into the following type of errors;
Error: Client error: Connection reset by peer Unable to retrieve next block transmission command. Number of already processed blocks: [210419].
Error: Client error: Connection reset by peer Unable to retrieve next block transmission command. Number of already processed blocks: [316108].
Is anyone familiar with this type of setup? Does this seem like a network type issue? Would moving from a micro instance to a larger instance at Amazon provide me with more network resources and resolve this issue?
Anyone care to take a stab?
I am trying to setup Veeam to use an Amazon EC2 instance as a backup repository and I am running into the following type of errors;
Error: Client error: Connection reset by peer Unable to retrieve next block transmission command. Number of already processed blocks: [210419].
Error: Client error: Connection reset by peer Unable to retrieve next block transmission command. Number of already processed blocks: [316108].
Is anyone familiar with this type of setup? Does this seem like a network type issue? Would moving from a micro instance to a larger instance at Amazon provide me with more network resources and resolve this issue?
Anyone care to take a stab?
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Backing up to Amazon EC2
Yes, usually this type of errors is caused by the unstable network connection.
Please include your support case ID number as explained when you click New Topic.
Please include your support case ID number as explained when you click New Topic.
-
- Novice
- Posts: 3
- Liked: never
- Joined: Sep 14, 2010 6:34 pm
- Full Name: Garufo
- Contact:
[MERGED] Deduplication Storage: "Connection reset by peer"
Hello,
We are having several backup fails in our environment (Case # 00134265). We have a Veeam Backup 6.1 patch 1 (virtual machine), and we use a Storage Deduplication Appliance (Fujitsu CS800) as a repository, attached by as a NFS resource to a Red Hat Linux virtual machine in the same VMware 4.1 environment.
The CS800 NFS resource is with the deduplication active. 2 weeks ago we had errors in the backups that we assumed that were about the Storage Array where the virtual machines are running because we had an incident in the storage. For this reason we had to reconfigure all the backup tasks in a no deduplication resource, all the jobs where running in the right way. Last week we where "moving" the backup files to a new repositoy in the CS800, another NFS resource in the Red Hat virtual machine, but with deduplication active. This weeked had started again the errors:
Error: Cliente error: Connection reset by peer. Unable to retrieve next block transmission command. Number of already processed blocks: [9811]
Busy: Source 13%> Proxy> 6%> Target 89%
(and similiar errors always, changing the number of processed blocks). Sometimes it stopped in a virtual disk, sometimes in another. There are 2 tasks at least with that error.
We had do the following troubleshooting:
- Disable multiple TCP/IP in global traffic throttling.
- Backup task options: Inline deduplication: disabled. Compression: NONE. Backup type: LAN
We are now again with Veeam support trying to find out what is happening.
Any suggest??
thanks in advanced
We are having several backup fails in our environment (Case # 00134265). We have a Veeam Backup 6.1 patch 1 (virtual machine), and we use a Storage Deduplication Appliance (Fujitsu CS800) as a repository, attached by as a NFS resource to a Red Hat Linux virtual machine in the same VMware 4.1 environment.
The CS800 NFS resource is with the deduplication active. 2 weeks ago we had errors in the backups that we assumed that were about the Storage Array where the virtual machines are running because we had an incident in the storage. For this reason we had to reconfigure all the backup tasks in a no deduplication resource, all the jobs where running in the right way. Last week we where "moving" the backup files to a new repositoy in the CS800, another NFS resource in the Red Hat virtual machine, but with deduplication active. This weeked had started again the errors:
Error: Cliente error: Connection reset by peer. Unable to retrieve next block transmission command. Number of already processed blocks: [9811]
Busy: Source 13%> Proxy> 6%> Target 89%
(and similiar errors always, changing the number of processed blocks). Sometimes it stopped in a virtual disk, sometimes in another. There are 2 tasks at least with that error.
We had do the following troubleshooting:
- Disable multiple TCP/IP in global traffic throttling.
- Backup task options: Inline deduplication: disabled. Compression: NONE. Backup type: LAN
We are now again with Veeam support trying to find out what is happening.
Any suggest??
thanks in advanced
-
- Chief Product Officer
- Posts: 31816
- Liked: 7302 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Unable to retrieve next block transmission command
Most likely, this is caused by network connection drops.
-
- Novice
- Posts: 3
- Liked: never
- Joined: Sep 14, 2010 6:34 pm
- Full Name: Garufo
- Contact:
Re: Unable to retrieve next block transmission command
Thank you, we are trying a troubleshooting about it,
thanks
thanks
-
- Influencer
- Posts: 22
- Liked: never
- Joined: Jan 27, 2012 12:25 pm
- Full Name: George
- Contact:
[MERGED] End of file Unable to retrieve next block transmiss
Hi Guys,
I have seen other posts regarding this error but was wondering if you could help me out.
I have a Veeam Server running on Windows with a Linux repository, I'm running on 6.1 and at the moment i get this error during the middle of the job. Its running just fine for a while then stops. Not sure what could be causing it everything on the network looks fine.
The errors message i get is: End of file: Unable to retrieve next block transmission command. Number of already processed blocks:
Any thoughts would be much appreciated!
Thanks!
I have seen other posts regarding this error but was wondering if you could help me out.
I have a Veeam Server running on Windows with a Linux repository, I'm running on 6.1 and at the moment i get this error during the middle of the job. Its running just fine for a while then stops. Not sure what could be causing it everything on the network looks fine.
The errors message i get is: End of file: Unable to retrieve next block transmission command. Number of already processed blocks:
Any thoughts would be much appreciated!
Thanks!
-
- Enthusiast
- Posts: 28
- Liked: 2 times
- Joined: Oct 05, 2012 6:36 am
- Full Name: Christer Sundqvist
- Contact:
Re: Unable to retrieve next block transmission command
Hi,
We experience the exact same error. have you found any resolution to the issue
MGD
We experience the exact same error. have you found any resolution to the issue
Code: Select all
Error: Client error: Connection reset by peer Unable to retrieve next block transmission command. Number of already processed blocks: [723354]
-
- VP, Product Management
- Posts: 27377
- Liked: 2802 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: Unable to retrieve next block transmission command
Hi Christer, what is the network connection to your repository? Are you using a WAN link?
-
- Enthusiast
- Posts: 28
- Liked: 2 times
- Joined: Oct 05, 2012 6:36 am
- Full Name: Christer Sundqvist
- Contact:
Re: Unable to retrieve next block transmission command
Hi
No it´s set to LAN Target.
No it´s set to LAN Target.
-
- VP, Product Management
- Posts: 27377
- Liked: 2802 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: Unable to retrieve next block transmission command
Yes, but what is your backup repository? Where is it located? According to the error message, the connection doesn't seem to be very stable.
-
- Enthusiast
- Posts: 28
- Liked: 2 times
- Joined: Oct 05, 2012 6:36 am
- Full Name: Christer Sundqvist
- Contact:
Re: Unable to retrieve next block transmission command
Hi
Our Backup Rep is a Linux gateway witch has mounted NFS share from our Datadomain box. I have seen the same error when we had the linux gateway had a NFS mount to a physical linux box. The backuprepositorys we use are virtual.
Christer
Our Backup Rep is a Linux gateway witch has mounted NFS share from our Datadomain box. I have seen the same error when we had the linux gateway had a NFS mount to a physical linux box. The backuprepositorys we use are virtual.
Christer
-
- Enthusiast
- Posts: 88
- Liked: 25 times
- Joined: Sep 25, 2012 7:57 pm
- Contact:
Re: Unable to retrieve next block transmission command
I am having this same error in 6.5 when backing up to our remote site over WAN. It happens on one or two servers per job. If I retry, sometimes it works, sometimes it doesn't.
Working with support, they advised to use network mode on the proxy for that job (i set up a new proxy in network mode, and pointed that job at it). I tried this over the weekend, and sure enough the job completed without error.
Curious to see if this actually fixes it. Will report back after a few more jobs run
Working with support, they advised to use network mode on the proxy for that job (i set up a new proxy in network mode, and pointed that job at it). I tried this over the weekend, and sure enough the job completed without error.
Curious to see if this actually fixes it. Will report back after a few more jobs run
-
- Enthusiast
- Posts: 49
- Liked: 1 time
- Joined: Jan 07, 2011 9:30 pm
- Full Name: Tim O'Pry
- Contact:
Re: Unable to retrieve next block transmission command
We have started seeing these errors since the upgrade to 6.5 (not before) and no other upgrades/changes other than the upgrade.
We receive this error primarily while backing up one VM, yet it shares the same SAN/iSCSI network as all of our 3 hosts and the frequency is never more than once per day, 4-5 days out of 7.
All backup sets are using SAN and other than this periodic issue, everything else works normally and test restores before/after this error show no problems. Veeam is installed withing a VM and the host and SAN are connected via dedicated gigabit switch for all iSCSI traffic with 6 ports on the SAN and 4 each per host. No other indications of network issues.
The specific error we receive is:
If anyone has any suggestions on what else to check/how to track this further.
We receive this error primarily while backing up one VM, yet it shares the same SAN/iSCSI network as all of our 3 hosts and the frequency is never more than once per day, 4-5 days out of 7.
All backup sets are using SAN and other than this periodic issue, everything else works normally and test restores before/after this error show no problems. Veeam is installed withing a VM and the host and SAN are connected via dedicated gigabit switch for all iSCSI traffic with 6 ports on the SAN and 4 each per host. No other indications of network issues.
The specific error we receive is:
Code: Select all
Error: Client error: End of file Unable to retrieve next block transmission command. Number of already processed blocks: [0]
-
- Enthusiast
- Posts: 88
- Liked: 25 times
- Joined: Sep 25, 2012 7:57 pm
- Contact:
Re: Unable to retrieve next block transmission command
I tried running in network mode, as suggested by support. It worked over the weekend, but last night's inc failed with the same error. So i am back to the drawing board.
This has only ever happened on my "offiste" job, which is across a WAN. The local job has been solid
This has only ever happened on my "offiste" job, which is across a WAN. The local job has been solid
-
- Chief Product Officer
- Posts: 31816
- Liked: 7302 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Unable to retrieve next block transmission command
If you are always gettin [0] at the end of the error, then I'd say this is definitely completely different issue (not what was being discussed in this thread above). WAN issues would cause this number to be random every time the job fails, because connection drop is random.
-
- Enthusiast
- Posts: 88
- Liked: 25 times
- Joined: Sep 25, 2012 7:57 pm
- Contact:
Re: Unable to retrieve next block transmission command
makes sense. my number is always some random blockGostev wrote:If you are always gettin [0] at the end of the error, then I'd say this is definitely completely different issue (not what was being discussed in this thread above). WAN issues would cause this number to be random every time the job fails, because connection drop is random.
-
- Chief Product Officer
- Posts: 31816
- Liked: 7302 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Unable to retrieve next block transmission command
In that case, this is clearly WAN link issue that did not exist before. In fact, 6.5 can sustain much worse packet loss than all previous B&R versions, so the issues must be real bad comparing to how this one link used to work when you had previous B&R versions installed. To be precise, this engine's enhancement was added in B&R 6.1 Patch 1.
-
- Enthusiast
- Posts: 88
- Liked: 25 times
- Joined: Sep 25, 2012 7:57 pm
- Contact:
Re: Unable to retrieve next block transmission command
we just bought Veeam, I wasn't using it for offsite backups before. I had a demo of 6.1, but didn't start doing WAN backups until after I installed 6.5Gostev wrote:In that case, this is clearly WAN link issue that did not exist before. In fact, 6.5 can sustain much worse packet loss than all previous B&R versions, so the issues must be real bad comparing to how this one link used to work when you had previous B&R versions installed. To be precise, this engine's enhancement was added in B&R 6.1 Patch 1.
The connection is a 25Mb cable modem at a branch office. Otherwise rock solid. Connected to our HQ via Cisco ASA site-to-site VPN. The VPN connection is rock stable (currently shows it has been established for 53d : 5h). The users at the remote site use several servers that reside in the HQ, as well as Cisco IP phones, where Call Manager servers are in the HQ. No problems at all with anything else.
I know cable internet can be spotty, but seems strange that only Veeam is having issue. I would expect to see the VPN connection flap if there were serious issues.
-
- Enthusiast
- Posts: 82
- Liked: 6 times
- Joined: May 01, 2012 3:00 pm
- Contact:
Re: Unable to retrieve next block transmission command
Hi Gostev,Gostev wrote:If you are always gettin [0] at the end of the error, then I'd say this is definitely completely different issue (not what was being discussed in this thread above). WAN issues would cause this number to be random every time the job fails, because connection drop is random.
I just got a few of these messages that ended with [0] for the same job over the weekend. My jobs are set to auto-retry, and this error does not come up the second time the job runs. Any ideas what might be causing it? My repository is a windows server and it's going over a WAN link via VPN. I monitor that link and don't show any packet loss or high latency.
*edit - forgot to mention I am using 6.5
thanks
Thanks,
Aaron
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Unable to retrieve next block transmission command
Aaron, have you already contacted support with this issue?
-
- Enthusiast
- Posts: 82
- Liked: 6 times
- Joined: May 01, 2012 3:00 pm
- Contact:
Re: Unable to retrieve next block transmission command
Hi foggy,foggy wrote:Aaron, have you already contacted support with this issue?
No I have not done that. I was hoping someone had a some background on this and I could save a little time
If that's what I should do I'll go ahead and open a case.
Thanks,
Aaron
-
- Enthusiast
- Posts: 88
- Liked: 25 times
- Joined: Sep 25, 2012 7:57 pm
- Contact:
Re: Unable to retrieve next block transmission command
I worked with support re: the issue and never got it resolved. WAN connection is the scape-goat. It usually completes during the retry, so i just live with it.
-
- Lurker
- Posts: 1
- Liked: never
- Joined: Mar 19, 2013 1:33 pm
- Full Name: RLPITS Ottawa
- Contact:
[MERGED] : Failed backups
Hi,
We've been having an issue with one of our backups. We have one backup for extreemly vital information that is set to run every hour that usually works but sporadically it fails with the error, Error: Client error: End of file Unable to retrieve next block transmission command. Number of already processed blocks: [0]. On occasion the houerly backups will fail consequtively 2 or three times in a row but usually the failiure occurs once a day.
This is on a LAN connection backing up a virtual SERVER2008 machine with the veeam backup software SERVER2008 running on the same ESX host backing up to another non-virtualized backup server running SERVER2008 on the same LAN
Need some insight on this matter
We've been having an issue with one of our backups. We have one backup for extreemly vital information that is set to run every hour that usually works but sporadically it fails with the error, Error: Client error: End of file Unable to retrieve next block transmission command. Number of already processed blocks: [0]. On occasion the houerly backups will fail consequtively 2 or three times in a row but usually the failiure occurs once a day.
This is on a LAN connection backing up a virtual SERVER2008 machine with the veeam backup software SERVER2008 running on the same ESX host backing up to another non-virtualized backup server running SERVER2008 on the same LAN
Need some insight on this matter
-
- Product Manager
- Posts: 20415
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Unable to retrieve next block transmission command
As it’s been already mentioned, this issue is almost certainly related to the network drops. So, if I were you, I would doublecheck it first.
Furthermore, feel free to open a ticket with our support team and let them take a closer look at your backup infrastructure.
Hope this helps.
Thanks.
Furthermore, feel free to open a ticket with our support team and let them take a closer look at your backup infrastructure.
Hope this helps.
Thanks.
-
- Veteran
- Posts: 367
- Liked: 41 times
- Joined: May 15, 2012 2:21 pm
- Full Name: Arun
- Contact:
Re: Unable to retrieve next block transmission command
I am getting the same error for the past two days for my exchange VM. However I have not observed any network drops. A case has been also opened for the same.
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Unable to retrieve next block transmission command
Arun, kindly please update this thread with the results of the investigation (as you usually do ), as we are very interested to hear the outcome. Thanks!
-
- Chief Product Officer
- Posts: 31816
- Liked: 7302 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Unable to retrieve next block transmission command
...or just include the support case ID so we could follow your support case.
-
- Veteran
- Posts: 367
- Liked: 41 times
- Joined: May 15, 2012 2:21 pm
- Full Name: Arun
- Contact:
Case # 00203616
Thanks Gostev..mentioned the case id above... I am sure support will definitely help in resolving this as always
-
- Veteran
- Posts: 367
- Liked: 41 times
- Joined: May 15, 2012 2:21 pm
- Full Name: Arun
- Contact:
Re: Unable to retrieve next block transmission command
Okay, support had asked me to use Wireshark to capture network packets from the source proxy. I am not sure if they could find anything from this.
However the job using a Linux proxy as a backup repository. This VM was using the VMXnet3 driver. Network drops were observed between the source proxy and the Linux proxy.
After changing the drivers from VMXnet3 to E1000, the job seems to be running proerly.
However it seems unknown the reason as to why the job used to work previously with the VMXnet3 driver and had to be changed.
Thanks!
However the job using a Linux proxy as a backup repository. This VM was using the VMXnet3 driver. Network drops were observed between the source proxy and the Linux proxy.
After changing the drivers from VMXnet3 to E1000, the job seems to be running proerly.
However it seems unknown the reason as to why the job used to work previously with the VMXnet3 driver and had to be changed.
Thanks!
-
- Veteran
- Posts: 367
- Liked: 41 times
- Joined: May 15, 2012 2:21 pm
- Full Name: Arun
- Contact:
Re: Unable to retrieve next block transmission command
Here the facts laid out by support which resolved the issue.
a) As per the logs sent the connection broke between the source and the target agent.
b) based on the setup we figured out, that the problem is most likely in the target.
c) based on the fact that I saw more issues with vmxnet3 NICs than E1000 I recommended to try switching the NIC type which has helped.
d) Now based on the Fact that changin the NIC type help, we can guess that most likely it is either the driver or the OS issue.
e) My next idea was to redeploy the linux VM from other distributive let's say CentOS and assign it a E1000 NIC. And see how it handles the incoming network IO.
Hope this helps,
Thanks
a) As per the logs sent the connection broke between the source and the target agent.
b) based on the setup we figured out, that the problem is most likely in the target.
c) based on the fact that I saw more issues with vmxnet3 NICs than E1000 I recommended to try switching the NIC type which has helped.
d) Now based on the Fact that changin the NIC type help, we can guess that most likely it is either the driver or the OS issue.
e) My next idea was to redeploy the linux VM from other distributive let's say CentOS and assign it a E1000 NIC. And see how it handles the incoming network IO.
Hope this helps,
Thanks
Who is online
Users browsing this forum: No registered users and 14 guests