-
- Enthusiast
- Posts: 48
- Liked: 5 times
- Joined: Aug 01, 2016 6:37 pm
- Full Name: Richard
- Contact:
Slow backup to tape - windows dedupe?
Good Afternoon, I'm trying to track down why I'm getting very slow transfer speeds recently. I run two D2D jobs -
Daily continuous incremental keeping 7 restore points and then a separate weekly full.
Then each day the daily is written to tape so it has to convert the chain into a full, and then once a week the weekly is copied to a separate tape pool.
Library is an MSL2024 with LTO6.
The daily tape run I get ~45MB/Sec, the weekly I get ~113MB/Sec.
I used to get much better performance on the daily - I didn't really notice the change in speed for a while but could it be because one of the VMs now has dedupe enabled?
thanks
Daily continuous incremental keeping 7 restore points and then a separate weekly full.
Then each day the daily is written to tape so it has to convert the chain into a full, and then once a week the weekly is copied to a separate tape pool.
Library is an MSL2024 with LTO6.
The daily tape run I get ~45MB/Sec, the weekly I get ~113MB/Sec.
I used to get much better performance on the daily - I didn't really notice the change in speed for a while but could it be because one of the VMs now has dedupe enabled?
thanks
-
- Product Manager
- Posts: 20413
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Slow backup to tape - windows dedupe?
By dedupe you refer to Windows Server Deduplication? If so, that can pretty much explain the difference in current and previous speeds, as now data needs to rehydrated prior to being archived. Thanks,
-
- Enthusiast
- Posts: 48
- Liked: 5 times
- Joined: Aug 01, 2016 6:37 pm
- Full Name: Richard
- Contact:
Re: Slow backup to tape - windows dedupe?
Hi, Ah ok - I was under the impression that veeam and windows dedupe complemented each other? https://www.veeam.com/blog/how-to-get-u ... ation.html or is that purely for the D2D?
thanks
thanks
-
- Product Manager
- Posts: 20413
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Slow backup to tape - windows dedupe?
Not sure whether I follow you here.
Backup to tape job takes data stored on disk and archives it to tape. Since in your case data is stored on disk in deduplicated state, it has to be rehydrated (or returned to its original state in other words) before it can be written to tape. Thus, the performance decrease.
Thanks.
Backup to tape job takes data stored on disk and archives it to tape. Since in your case data is stored on disk in deduplicated state, it has to be rehydrated (or returned to its original state in other words) before it can be written to tape. Thus, the performance decrease.
Thanks.
-
- Enthusiast
- Posts: 48
- Liked: 5 times
- Joined: Aug 01, 2016 6:37 pm
- Full Name: Richard
- Contact:
Re: Slow backup to tape - windows dedupe?
Hi - so backup to disk is deduped, but to tape is not? But otherwise compressed? Is there not a way of keeping it deduped?
thanks
thanks
-
- Product Manager
- Posts: 20413
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Slow backup to tape - windows dedupe?
Getting back to original question - by dedupe you refer to Windows Server Deduplication or VB&R backup job one?
-
- Enthusiast
- Posts: 48
- Liked: 5 times
- Joined: Aug 01, 2016 6:37 pm
- Full Name: Richard
- Contact:
Re: Slow backup to tape - windows dedupe?
Windows, yes. We have an enourmous amount of very similar files that aren't accessed often but need to be available - so it works very well for us saving nearly a terabyte of space.
-
- Product Manager
- Posts: 20413
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Slow backup to tape - windows dedupe?
Correct, because Windows Server Deduplication mechanism does not cover tapes, only files stored on a file system.So backup to disk is deduped, but to tape is not?
Data can be compressed by your tape device, if it supports that.But otherwise compressed?
Nope, as I've mentioned already, data needs to be rehydrated before being written to tapes.Is there not a way of keeping it deduped?
Thanks.
-
- Enthusiast
- Posts: 48
- Liked: 5 times
- Joined: Aug 01, 2016 6:37 pm
- Full Name: Richard
- Contact:
Re: Slow backup to tape - windows dedupe?
Sorry for the delay in replying. I think my issue is a combination - when I run the weekly backup that does a full then copies it to tape. Currently 5 hours to disk and then 6 hours for the copy to tape.
Whereas the daily incremental takes half an hour to save the changed data, then over 6 hours for the merge. Then a further 9 hours to send a full backup to tape.
The data is the same so it's the method that's adding 4 hours to the daily backup. I think I need to invest in a load more disk space and change to a full daily backup instead of incremental as well?
Whereas the daily incremental takes half an hour to save the changed data, then over 6 hours for the merge. Then a further 9 hours to send a full backup to tape.
The data is the same so it's the method that's adding 4 hours to the daily backup. I think I need to invest in a load more disk space and change to a full daily backup instead of incremental as well?
-
- Product Manager
- Posts: 20413
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Slow backup to tape - windows dedupe?
I'm slightly confused by your current jobs' implementation and goal you're trying to achieve. So, correct me if I'm wrong:
- You have two backup jobs (daily and weekly ones)
- Both process the same set of VMs
- Both use the same mode (forward incremental with active full)
- Both are pointed to the same repository
- Both get archived to tapes
- Jobs get archived by different backup to tape jobs
- One tape job that processes weekly backup job copies only full backups
- Another tape job that processes daily backup job copies both full and incremental backups
- Incremental cycle of tape job that processes daily backup job much longer that full cycle of tape job that process weekly backup job?
(And deduplication is enabled where? Repository server, backup server, inside VM that gets backed up by two backup jobs?)
Thanks.
- You have two backup jobs (daily and weekly ones)
- Both process the same set of VMs
- Both use the same mode (forward incremental with active full)
- Both are pointed to the same repository
- Both get archived to tapes
- Jobs get archived by different backup to tape jobs
- One tape job that processes weekly backup job copies only full backups
- Another tape job that processes daily backup job copies both full and incremental backups
- Incremental cycle of tape job that processes daily backup job much longer that full cycle of tape job that process weekly backup job?
(And deduplication is enabled where? Repository server, backup server, inside VM that gets backed up by two backup jobs?)
Thanks.
-
- Enthusiast
- Posts: 48
- Liked: 5 times
- Joined: Aug 01, 2016 6:37 pm
- Full Name: Richard
- Contact:
Re: Slow backup to tape - windows dedupe?
Hi, sorry if I've been unclear - still quite new to backups in general and Veeam. I've added notes below
v.Eremin wrote:I'm slightly confused by your current jobs' implementation and goal you're trying to achieve. So, correct me if I'm wrong:
- You have two backup jobs (daily and weekly ones) - YES
- Both process the same set of VMs - YES
- Both use the same mode (forward incremental with active full) - NO - the daily runs forward incremental keeping 5 restore points, the weekly is a simple full
- Both are pointed to the same repository - YES
- Both get archived to tapes - YES
- Jobs get archived by different backup to tape jobs - YES - as you don't have a daily GFS option
- One tape job that processes weekly backup job copies only full backups - YES
- Another tape job that processes daily backup job copies both full and incremental backups - NO - it builds a full from the incrementals (each incremental is ~20-30GB so not huge)
- Incremental cycle of tape job that processes daily backup job much longer that full cycle of tape job that process weekly backup job? - YES
(And deduplication is enabled where? Repository server, backup server, inside VM that gets backed up by two backup jobs?) - On one inside VM.
Thanks.
-
- Product Manager
- Posts: 20413
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Slow backup to tape - windows dedupe?
Ok, so it's full backup cycle in both cases (not incremental versus full): daily job synthesizes virtual full backup directly on tapes, weekly one copies full backup to tapes. Due to the differences in nature (direct copy against synthesized restore point creation) those jobs cannot be compared fairly.it builds a full from the incrementals (each incremental is ~20-30GB so not huge)
You can switch daily job to reversed incremental mode, so that latest restore point (which is always a full backup) gets directly archived to tapes on daily basis. Otherwise, if the proposed scenario is not applicable, you might want to open a ticket with our support team and investigate whether the performance of virtual full backup can be improved somehow.
Then, disregard what I've written previously, since I was talking about enabling deduplication on top of backup repository. Usage of deduplication inside VM should result in bigger increments, that's it.(And deduplication is enabled where? Repository server, backup server, inside VM that gets backed up by two backup jobs?) - On one inside VM.
Thanks.
-
- Enthusiast
- Posts: 48
- Liked: 5 times
- Joined: Aug 01, 2016 6:37 pm
- Full Name: Richard
- Contact:
Re: Slow backup to tape - windows dedupe?
Hi,
Thank you - I've now removed dedupe from the server and whilst it's costing me a lot more disk space, I'm now down to sub-3 hour runs so I'll take that!
Thanks
Thank you - I've now removed dedupe from the server and whilst it's costing me a lot more disk space, I'm now down to sub-3 hour runs so I'll take that!
Thanks
-
- Enthusiast
- Posts: 48
- Liked: 5 times
- Joined: Aug 01, 2016 6:37 pm
- Full Name: Richard
- Contact:
Re: Slow backup to tape - windows dedupe?
Hi All,
I have one more issue to solve - I have my weekly and monthly GFS tape pools. Both copy a weekly full D2D backup.
Weekly works fine, but for some reason the monthly runs twice - once on the first Sunday of the month as scheduled, but then it runs again the next day (and correctly finds no new files so doesn't do anything) Any idea how I can trace why this is doing this?
Thanks
I have one more issue to solve - I have my weekly and monthly GFS tape pools. Both copy a weekly full D2D backup.
Weekly works fine, but for some reason the monthly runs twice - once on the first Sunday of the month as scheduled, but then it runs again the next day (and correctly finds no new files so doesn't do anything) Any idea how I can trace why this is doing this?
Thanks
-
- Product Manager
- Posts: 14726
- Liked: 1706 times
- Joined: Feb 04, 2013 2:07 pm
- Full Name: Dmitry Popov
- Location: Prague
- Contact:
Re: Slow backup to tape - windows dedupe?
Do you have all the source restore points in place for the Monthly backup date? GFS job can restart itself if source jobs are running.
-
- Enthusiast
- Posts: 48
- Liked: 5 times
- Joined: Aug 01, 2016 6:37 pm
- Full Name: Richard
- Contact:
Re: Slow backup to tape - windows dedupe?
I believe so - it uses the same source as the weekly (which I've spotted also runs twice sometimes) and that only takes 4 hours or so.
I've finally got the last 10Gbe link in so everything should run quite a bit faster now so will re-jig the timings and see if that helps.
I've finally got the last 10Gbe link in so everything should run quite a bit faster now so will re-jig the timings and see if that helps.
Who is online
Users browsing this forum: No registered users and 9 guests