Issues with Veeam Backup - Oracle?

[russwatkins]

Hi,

I have been running Veeam for a year now - no major issues and the product is superb!

I did however come across a problem earlier in the year. The Veeam backups were left scheduled to run one evening whilst an upgrade was happening on a VM (Oracle upgrade from 9i to 10). The backups kicked in whilst the migration was happening and the snapshot took around 14 hours to remove! Consequently we had one or two other issues and decided to take this server out of the Veeam backups for now.

Last night, this issue arose again on one of our other servers - I came in to a none responsive VM. When I looked at the backups, the server backed up at 500k per second (this VM usually backups up at 10 times this speed as it is network connected, not on our FC backbone). Consequently the snapshot was trying to remove and taking a long time. In the end the backup competed in 6 hours which would usually take around 30 minutes.

The only similarity between the two systems is that they have Oracle 9i on them. I don't currently utilise any pre-post scripts and just back up "as-is". The backup was a full that was running - server is Windows 2003 Enterprise. Could this be a factor? The backups have run for several months with no issues.

Our site DBA is currently wanting Veeam backups removed from all our Oracle servers, so I need some proof that Veeam can back up these servers correctly!

[chrisdearden]

how much changed data was on the VM? If someone does something during that backup to cause a lot of change to the VM , then the snapshot will grow as you saw with the upgrade. When the backup slowed down , what was the bottleneck showing as?

[Vitaliy S.]

Last night, this issue arose again on one of our other servers - I came in to a none responsive VM. When I looked at the backups, the server backed up at 500k per second (this VM usually backups up at 10 times this speed as it is network connected, not on our FC backbone). Consequently the snapshot was trying to remove and taking a long time. In the end the backup competed in 6 hours which would usually take around 30 minutes.

The backup was a full that was running - server is Windows 2003 Enterprise. Could this be a factor? The backups have run for several months with no issues.

Well...how much time does your usual full backup of this VM take? "500k per second" is an unacceptably low performance rate, I wonder what does your job log say about that. Also If your backups were running fine for months, then I doubt that your current approach has some pitfalls... we just need to verify why your job was so slow.

Btw, what was the size of the VM snapshot when it started its consolidation? Do you have any monitoring tool to look after VM snapshot size and age?

[russwatkins]

Hi,

Screenshots of a working backup from last week and the failed one from last night.

HOWEVER........i've just found out that a change to the Oracle database on this server was made yesterday - looking at the esx logs the disk activity on this host went up quite a lot - this is likely to have had an impact I would have thought. Although would the snapshot removal cause the server to become unresponsive for a couple of hours?

[Vitaliy S.]

Although would the snapshot removal cause the server to become unresponsive for a couple of hours?

If VM snapshot is very large, then I guess it could be the reason why your VM got stunned during snapshot consolidation process.

Please take a look at this topic for some tips that might help you in this situation: Large VM Backups go to 100%, then fail when removing snapshot

[chrisdearden]

I wonder if the host management network wasn't happy there either. As it is a standalone host , why not run a windows 7 Vm on the same host and use it as a hotadd proxy ?