Veeam B&R 7 patch4 : new job stucked in the middle

[omerlin98000]

Hello to all (this is my very first post, but old Veeam user)

I have a VMware infrastructure (vCenter 5.5U1 but 3 ESX servers in 4.1GA - in process to be upgraded to 5.1U2 in a few weeks).

New Veeam Backup installation, 7.0 upgraded to patch 4 (7.0.0.781).
All backup jobs created from scratch.
4 proxies (including own Veeam B&R server), up to 10 simultaneous jobs. No throttle, all VM in the same network/subnet/switches.
Backup from two NetApp NAS to one QNAP.

Until now, 5 jobs running ok for 8 VM since a week.
Just added two days ago a new job to backup a SQL server (server in AD, credentials ok, VSS ok).
The backup job start approx. around 23:05 (after the previous other 5 jobs). It's scheduled after all others, so it's alone.

The first day, C: (system, 50GB) and E: (logs, 200GB) have been backed-up (hot-add on three of my four different proxies), I had to cancel the D: (data, 500GB) around 14:15 as the job seemed to do nothing at all.
This night, same problem but slightly different: D: (data, 500GB) is ok, but C: and E: seems to do nothing.
I see 77% about this this job, nothing advancing. In the throughput window, I see a very tiny read each few minutes, and if I switch to the all time view, that's interesting, I see big throughput from 23:06 (beginning) to approx 01:13, then no more throughput until the time I write this post.

Very strange !

If someone have seen this kind of problem or has a clue, I'm interested in...

[Dima P.]

Hello Olivier,

Was “application-aware image processing” and to “truncate transaction log files” enabled for SQL backup job?

[omerlin98000]

Yes it was (2008R2, Tools ok), VSS seems ok, and I have the default option to truncate logs after successful backup.
I have just cancelled a second time this job a few minutes ago (still at 77% as this morning).

Not sure it's important, but initially, I had two other SQL VM to backup in the same job, but their credentials were not OK, so I had one VM SQL backup job starting well, and two VM failing immediately for bad credentials in this very same job.
I can't see any logical influence between the 2 failing VMs and the one that starts ok and be stucked after a few hours.
Nevertheless, I have removed the two other VM from this job, so we will know if they were responsible for something: tonight, the job is scheduled to do only this SQL VM.

OR SHOULD I DELETE THE INCONSISTENT ACTUAL BACKUP (0 restore points), DELETE AND RECREATE THE JOB ???
No problem if I have to do that, because I have no consistent backup with this new v7 job.

Here the first run with two disk ok and the third very long until cancellation :
The same in second run this night, only one disk ok, also cancelled by me a few minutes ago (you can see the throughput) :

[Dima P.]

Yes, it seems to be a preferable solution to delete the old backup, re-create the whole job from scratch and let us know how it goes. Thank you.

[omerlin98000]

Done, old inconsistent backup files and backup job deleted, job recreated with this only VM
Keep you informed.

[foggy]

Olivier, what if you try to backup this VM using network mode?

I also recommend contacting support in case the issue persists (as it is recommended to do with every technical issue).

[omerlin98000]

Thanks Dmitry and foggy

Here is the status of the newly created job for the one SQL server (remark: the two other SQL servers with also a new job and correct credentials have been backuped correctly this night, see one line "SQL autres" above faulty BWS-SQL01 job):



Same player shoot again ??? Not exactly
The job start at 22:54, and you can see in throughput that at 00:10, there is no more traffic/throughput, BUT the processed percentage, this time, is moving forward, 2% the time I spent to write this post !!!
Hard disks 2 and 3 have few data yet: 67GB/500GB and 15GB/200GB respectively, and have been initialized in thin format: is this that the job have difficulty to read "empty space" ???

foggy, I haven't seen your Network mode advice before this new job, so I will let the job finished, and I will see what happen this night...

[omerlin98000]

The jump from 29% to 31% was purely random at the time I check.

See the throughput graph:


It seems that the job is now reading a little bit of disks every two minutes, and nothing...
At this rythm, job will be finished in a few days

foggy: I missed something about Network mode, isn't it for replication jobs ? Where do I find this setting in backup job ???

[Shestakov]

I missed something about Network mode, isn't it for replication jobs ? Where do I find this setting in backup job ???

Olivier, there are 3 Transport Modes: (Network, Virtual Appliance and Direct SAN).

Transport mode is a method that is used by the Veeam transport service to retrieve VM data from the source host and write VM data to the target destination. It can be defined in the settings of the backup proxy that performs the job. All 3 modes do the same, but with different configuration and performance.

FAQ can also be helpful. Thanks

[omerlin98000]

Shame on me , Nikita, just forgot that I had to set this in the proxy configuration !!!
So I will transform one of my proxies (all are on Automatic) to a Network transport mode proxy, and force this job to go through this proxy tonight.

[Shestakov]

No problem, Olivier!
Let us know how it will work. Thanks.

[omerlin98000]

The network mode for this particular proxy/job solved the problem



Many thanks for the tip, foggy, and to all for your answers.

I will try to go back to automatic transport mode for tonight, perhaps this incremental job (with weekly full synthetic) will work better with increments ???
If not, I will keep one of my proxies as network transport mode proxy, just for that kind of problems...

[foggy]

I'm glad network mode solved the performance issue, now we know that the problem relates to hotadd. If you would like to investigate it deeper, feel free to contact support.

[omerlin98000]

FYI, the next day job (which have to deal with the same disks but incremental mode) has finished in hotadd mode in less than 6 minutes, so the throughpput problem was only linked to the FIRST full job (problem solved by putting the proxy in network mode for this very first run)