First V10 issue: Tape backup switching between objects

[mkretzer]

Hello,

Case 04050943

so today we have the first V10 issue after all. We have a GFS tape job backing up > 1000 VMs in per-VM chains.

We wondered why the speed is so bad - it seems that after each object it takes some time to switch to the next one. So there are times when both drives do nothing because Veeam has to switch to the next Object and it takes a few seconds!
We had a similar issue with 9.5 where there where some database "pessimations".

Markus

[Dima P.]

Hello Markus,

I've asked QA folks to review the case details. Thank you!

[mkretzer]

I just did a SQL Trace... The following statement is taking 180 seconds and is doing *198263827* logical reads!!!
exec AggregateSqlPointsInfo

Also (perhaps more Tape-Specific) the following is taking 68 seconds and doing 18573300 logical reads....

Code: Select all

SELECT DISTINCT rk.* FROM [dbo].[CryptoKeys] rk
		INNER JOIN [dbo].[CryptoKeysRecoveryRecs] r ON rk.id = r.recovery_key_id 
		INNER JOIN [dbo].[CryptoKeys] k ON k.id = r.encrypted_key_id
			WHERE   k.[backup_id] = @backup_id
		UNION 
		SELECT DISTINCT rk.* FROM [dbo].[CryptoKeys] rk
		INNER JOIN [dbo].[CryptoKeysRecoveryRecs] r ON rk.id = r.recovery_key_id 
		INNER JOIN [dbo].[CryptoKeys] k ON k.id = r.encrypted_key_id
			WHERE EXISTS(SELECT 1 FROM [dbo].[Backup.Model.Storages] s WHERE s.[storage_key_id] = k.[id] AND s.[backup_id] = @backup_id
			UNION ALL 
			SELECT 1 FROM [dbo].[Backup.Model.Storages] s WHERE s.[meta_key_id] = k.[id] AND s.[backup_id] = @backup_id
			)

One more, 12 Seconds, 177725 logical reads
exec GetMetaInfoByBackupId @backup_id='xxxxxxxxxxxxxxxxxxxxxxxxxxxxx',@usn=0,@is_old_meta=0

Keep in mind our Veeam Server is dedicated, newest CPU, SSD, 768 GB RAM all for SQL. I wonder how this would work at all on lesser hardware.

[Dima P.]

Virtual full backup to tape could be the reason. Can you please check if virtual full backup was created or not for some machines in the list (should be noted in the task log)?

[mkretzer]

Hello,

no. We do the backup every sunday 23:30 *after* all synthetics have been created. So it just needs to read the VBKs from disk...

Markus

[Dima P.]

Thanks for the update Markus. I've already passed case details and your feedback to the QA team.