Comprehensive data protection for all workloads
Post Reply
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Defragment & compact full backup, how much free space?

Post by HendersonD »

I have my backup job set to "Defragment and compact full backups" once a month
I have my backup repository set to "Use per-VM backup files"
My repository is 8TB with 5TB used and 3TB free

How much free space does the defragment and compact operation need?
Does it need as much free space as what is currently being used (5TB)? In this case I would need to increase the repository to something greater than 10TB
Does it need as much free space as the largest VM that is backed up? My largest single VM is 1.8TB
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Defragment & compact full backup, how much free space?

Post by PTide »

Hi,

It needs as much free space as the largest VM's backup, 1.8 TB.

Thanks
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Defragment & compact full backup, how much free space?

Post by foggy »

In v9.5 it will require as much free space as all full VM backup files consume, since v9.5 will introduce parallel compact operations for the backup job (backup copy compact is already works parallel in v9).
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Re: Defragment & compact full backup, how much free space?

Post by HendersonD »

Currently I am consuming 5TB of space on my backup repository. So to prepare for the hopefully soon to be released v9.5 I should increase this backup repository to 11 or 12TB, correct? This is needed since v9.5 has parallel processing for the defrag/compact operation
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Defragment & compact full backup, how much free space?

Post by foggy »

This will be the upper free space estimate, in fact it might need less, since smaller VMs will be completed and release space faster.
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Re: Defragment & compact full backup, how much free space?

Post by HendersonD »

Since this is a little hard to predict and a dynamic process, to be on the safe side I will make this backup repository 11TB. I have two very large VMs, both file servers. If the parallel defrag/compact process happens to take place at the same time for both of these and a few other VMs then I want to make sure I have enough space.

How many parallel defrag/compact operations can take place at once?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Defragment & compact full backup, how much free space?

Post by foggy »

According to the repository concurrent tasks limit.
HendersonD
Expert
Posts: 158
Liked: 8 times
Joined: Jul 23, 2011 12:35 am

Re: Defragment & compact full backup, how much free space?

Post by HendersonD »

I backup about 40 VMs and my backup repository is set to 8 concurrent. To be on the safe side I will increasing this backup repository to 11TB so I do not have to worry about running out of space during defrag/compact once v9.5 is released.

Any word when 9.5 goes GA? We are also a Nimble shop and I am looking forward to the integration
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Defragment & compact full backup, how much free space?

Post by foggy »

We are planning to release it in October.
yakov@egtrade.co.il
Lurker
Posts: 2
Liked: never
Joined: Oct 20, 2016 1:37 pm
Full Name: Yakov Shtoots
Contact:

Re: Defragment & compact full backup, how much free space?

Post by yakov@egtrade.co.il »

foggy wrote:In v9.5 it will require as much free space as all full VM backup files consume, since v9.5 will introduce parallel compact operations for the backup job (backup copy compact is already works parallel in v9).
Hi,

Do i need to configure anything for parallel compact operation or it's done out of the box?
veremin
Product Manager
Posts: 20271
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Defragment & compact full backup, how much free space?

Post by veremin »

Out of the box. Thanks.
mcz
Veeam Legend
Posts: 835
Liked: 172 times
Joined: Jul 19, 2016 8:39 am
Full Name: Michael
Location: Rheintal, Austria
Contact:

Re: Defragment & compact full backup, how much free space?

Post by mcz »

Can you switch off paralell processing for compact operations? I know that I could decrease the number of concurrent tasks for the repo but that would also affect backup, etc. We have quite a slow repository and I think doing parallel processing for compact operations is slower then using the sequential approach. Thanks!
veremin
Product Manager
Posts: 20271
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Defragment & compact full backup, how much free space?

Post by veremin »

Nope, you cannot enable/disable parallel processing for specific operation. You can either disable it altogether or stick to limiting number of concurrent tasks. Thanks.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Defragment & compact full backup, how much free space?

Post by foggy » 1 person likes this post

Another way of limiting I/O activity is repository ingestion rate.
backupquestions
Expert
Posts: 186
Liked: 21 times
Joined: Mar 13, 2019 2:30 pm
Full Name: Alabaster McJenkins
Contact:

Re: Defragment & compact full backup, how much free space?

Post by backupquestions »

If I am using REFS repository, I understand it won't really defrag but would still "compact". Is there still a big space requirement or does the temporary new full file benefit from block cloning etc? I've got a powerful server with 16 concurrent task limit backing up a hundred VMs, so if the 16 vms that are parallel compacted happens to include some of the larger multi terabyte VMs then I would be in trouble if it indeed still needs a ton of space.
veremin
Product Manager
Posts: 20271
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Defragment & compact full backup, how much free space?

Post by veremin »

does the temporary new full file benefit from block cloning etc?
Yes, it does. Thanks!
Mgamerz
Expert
Posts: 159
Liked: 28 times
Joined: Sep 29, 2017 8:07 pm
Contact:

Re: Defragment & compact full backup, how much free space?

Post by Mgamerz »

Not to hijack the thread, but if I do a synthetic full, does that "compact" the backup since it's building a new full? I have never turned on defrag + compact as all my repo's are ReFS with fast clone, and I was under the assumption I don't need that (I do forward incremental forever with synthetic full and health checks).
veremin
Product Manager
Posts: 20271
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Defragment & compact full backup, how much free space?

Post by veremin » 1 person likes this post

Compact full functionality is unnecessary when periodic (active or synthetic) full is enabled. Thanks!
Mgamerz
Expert
Posts: 159
Liked: 28 times
Joined: Sep 29, 2017 8:07 pm
Contact:

Re: Defragment & compact full backup, how much free space?

Post by Mgamerz » 1 person likes this post

Cool, thanks!
fgw
Enthusiast
Posts: 84
Liked: 2 times
Joined: Jun 11, 2009 8:39 pm
Full Name: Franz Glatzer
Contact:

Re: Defragment & compact full backup, how much free space?

Post by fgw »

veremin wrote: Jul 23, 2019 6:06 pm Compact full functionality is unnecessary when periodic (active or synthetic) full is enabled. Thanks!
just to clarify:

when doing reverse incremental, the option Defragment and compact full backup file in Maintenance tab under Advanced Settings is not required, as synthetic full performed by reverse incremental will do this anyway?

could u please confirm!
veremin
Product Manager
Posts: 20271
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Defragment & compact full backup, how much free space?

Post by veremin »

Nope, in case of reverse incremental mode, compact full is still needed, unless you enable periodic active full backup. Thanks!
Post Reply

Who is online

Users browsing this forum: No registered users and 248 guests