calculating backup size

[janpaulsmit]

We use Veeam 7, and thanks to the new backup copy job feature we can use GFS-rotation, but I'm wondering how many full backups the backup copy job will create in the folowing situation:

Backup scheme:
* every 2 hours: 12 restore points
Backup copy:
* once a day: 14 restore points
* Weekly: 4 restore points
* monthly: 12 restore points
* yearly: 5 restore points

How many full backups wil I have? It's important to know to calculate the required disk space by using this formula:

Backup size = C * (F*Data + R*D*Data)
Data = sum of processed VMs size (actually used, not provisioned)
C = average compression/dedupe ratio (depends on too many factors, compression and dedupe can be very high, but we use 50% – worst case)
F = number of full backups in retention policy (1, unless backup mode with periodic fulls is used)
R = number of rollbacks (or increments) according to retention policy (14 by default)
D = average amount of VM disk changes between cycles in percent

Data=11TB and will grow to 22TB in 3 years

Thanks!

[Gostev]

1 + 4 +12 + 5 = 20 full backups in 5 years.

[janpaulsmit]

Thank you for your quick reply!

Just to summarize and to be sure, I put down the following calculation based on 11TB VM-size, 50% dedup/compression and 5% change rate

'Normal' backup:
Backup size = C * (F*Data + R*D*Data) (every 2 hours 12 restore points)
Backup size = 50 * (11*5 + 12*1*5)= 8,8TB (approximately)

backup copy:
Backup size = C * (F*Data + R*D*Data) (1 daily with 14 restore points, 4 week, 12 months, 5 years= (1+4+12+5) 22 full backups)
Backup size = 50 * (11*5 + 14*22*5)= 124TB (approximately)

Do you think this is a realistic calculation?

[foggy]

Jan, before answering your question, could you please clarify the following:
- What is the backup mode for the normal job? Forward or reversed incremental?
- What is the expected average amount of VM disk changes between cycles in percent (D=?) If I can derive it correctly from the 3 year 22TB estimation, it is ~1%, is that correct?

[janpaulsmit]

The backup mode is reversed incremental.
The customer gave us these figures: 11TB diskspace use now and expected grow to 22TB in three years. So the 1% would be correct.

[foggy]

Well, since this formula reflects the amount of space required at the specific moment in time (and does not reflect the fact that the size of the full backup (Data) increases with time), you could calculate the maximum space required after 5 years of running both jobs and it will be something like that (actual space will be less):

'Normal' backup:
Backup size = C * (F*Data + R*D*Data) (every 2 hours 12 restore points)
Backup size = 50 * (11*5 + 12*1*5)= 8,8TB (approximately)

50%(1*22+12*1%*22)=12.3TB

backup copy:
Backup size = C * (F*Data + R*D*Data) (1 daily with 14 restore points, 4 week, 12 months, 5 years= (1+4+12+5) 22 full backups)
Backup size = 50 * (11*5 + 14*22*5)= 124TB (approximately)

50%(22*22+14*1%*22)=243.5TB

[AdrianHinton]

Hi, Can I please jump onto this one and as the same question but only about the copy job and with no Deduplication on the target?

VM Current Size = 995GB

Backup Job size 1.4TB
Backup is Incremental
Backup Job Retention
Restore Points to keep = 91

Copy Job Retention
Restore Points to keep = 91
Weekly Backups = 3
Monthly Backkups = 11
Quarterly = 0
Yearly = 1

Thank you!

[foggy]

Adrian, I need to ask a few clarifying questions:

- do you need to calculate the amount of space required for both normal and backup copy jobs?
- what is the average amount of changes inside you VMs in percent?
- is periodic synthetic or active full scheduled for the normal backup job?
- what is the "Copy every" interval for the backup copy job?
- what do you mean by "no deduplication on target"? Do you mean raw storage on backup repository or you're saying Veeam dedupe is disabled for the backup copy job?

Thanks.

[AdrianHinton]

Hi,

I have been searching for the definitive way to calculate how much storage I need for backup copies. I can't seem to find anything truly useful and I am surprised that I can't, especially considering how widely used this product is. All of the forum posts that I have read seem only intelligible to those who have asked their specific questions based on their specific configurations. Is there a utility out there that one can use to determine how much space would be required over time in a backup-copy repo? What is the formula for any given scenario?

I have a VM, VeeamOne reports the change rate, VMWare tells me the VM size, I get to decide how many RP's I want to keep, the type of backup it will be and the archival retention. The disk in the repo is either Windows dedup enabled or not; How do I put all that together to know how much storage I will need available for 12 months, 24 months, 36 and so on?

I would really like to write a program so I can input a few values in and it can calculate or estimate these storage requirements for me. I would have though Veeam have already done that but my searches are proving useless in this area.

Looking forward to hearing from you and thank you for your support!

[foggy]

Adrian, there's no such a tool, however knowing the values you're talking about, you could perform a rough estimate of the space required using the formula and examples given above.

Hope this helps.

[AdrianHinton]

Hi and thank you for the reply. Could you help me understand the formula with the below values?

VMSizeGB = 1007.51GB
AVGChangeRateGBPerDay = 166.43GB (Reported by VeeamOne against the BackupJob)
BackupType = Incremental
RP's to keep = 7
Weekly = 3
Monthly = 11
Quarterly = 0
Yearly = 1
TargetDisk Dedup savings = 50%

Thank you!

[dellock6]

Adrian,
based on the rule of thumb of 50% reduction (conservative assumption to keep you on the safe side), you will have:
Full backup 500Gb
Incremental backup 83 Gb

So, with a 7 days incremental backup you will end up in reality to have 2 weeks of backup files, made of 2 fulls and 12 incremental. This means the storage usage will be 2 Tb of space for regular retention.

For GFS retention this is more simple, you simply count 500Gb and multiply it for the number of restore points, since all of them are fulls. So, again in your case it will be 15 restore points, that is 7,5 TB for GFS.

Luca.

[AdrianHinton]

Adrian,
For GFS retention this is more simple, you simply count 500Gb and multiply it for the number of restore points, since all of them are fulls. So, again in your case it will be 15 restore points, that is 7,5 TB for GFS.

If I look at my repo for a particular job and see a size for my latest VBK, I can add up the restore points to keep, multiply by the size of the latest VBK and that will give me my GFS storage requirements?

Latest VBK for a job = 365GB
GFS
Weekly = 3
Monthly = 11
Yearly = 1
Total = 15

Does it follow that for a year the GFS retention will equal 3+11+1*365GB/1024=5.34TB? And double that for 2 years?

If so, how does Server2012 dedup come into play when we are planning the physical storage for this? Will that yearly 5.34TB be deduped 50% (conservative estimate) and require that I have a minimum of 2.67TB of storage for a year just for GFS? Or will dedup do a better job on all these vbk's because they contain similar data?

I'm really confused as to how a company plans for VB&R storage. And surprised there is not a utility to calculate the requirements based on the way one wants to backup and archive.

[dellock6]

No double, on a 2 year base you will only add an additional yearly restore. Monthly 11 for example means keeping last 11 monthly restore points, no matter if you then go for multiple years. So the count will be 3+11+2=16.
Also, consider some additional space due to the fact your infrastructure will grow during time, and 1 year from now your VBK could be bigger in size.
Finally, windows dedupe will probably go well beyond 50%, considering as you correctly guessed the content of VBK files is similar, but the actual results could vary. You can look in other threads in these forums where other users have posted their results.

[foggy]

For those who are interested in assessing disk space required for backup repository, here's a handy calculator based on the formula above (created by Timothy, one of our systems engineers).

[blodsbror]

Hi,

I'm in the process of creating my first backup copy jobs. I have read all documentation, however am still a little unclear when it comes to calculating approx storage requirements for these jobs.
We have approx 10TB of full backups, with approx 700GB of change data for each incremental over 7 days of retention (approx 14.2TB for the week). I'm using the new forward incremental job type.

I have setup a copy job with GFS retention. Weekly (once per week kept for 4 weeks). Monthly (once per month kept for 3 months). Quarterly (once per quarter kept for 3 quarters). Yearly (first day of year, kept for 1 year).

If I calculate correctly, I should have 11 backup copy retention points in total for 1 year. Does each weekly, monthly, quarterly and yearly retention point require the approx 14.2 TB of data I generate in a week - approx 156.2 TB ? Or is similar data just reused for other retention points ?

Thanks!

[foggy]

Evan, for the entire year you will have 11 separate GFS restore points, which are stored as a separate full backup each (probably less, depending on their schedule, as the same restore point can be marked both as weekly and monthly, for example, if both are scheduled for the same day), plus the regular chain consisting of the full and a number of increments specified in retention settings. To get more precise estimate, I recommend to put in your numbers in the calculator mentioned in this thread above. Thanks.

[JaxIsland7575]

I am looking at spinning up a new site with VBR. Up to now I've used my own guessing and a lot of luck to estimate storage capacities. Is there an algorithm or tool that I can use to look at a Hyper-V environment and find a fairly accurate guess as to how much storage space I would need for a 14 day retention cycle? Thanks for any help!

Cheers!

[nielsengelen]

Have a look at http://rps.dewin.me. This will help you with your calculations.

[foggy]

Also, above you can find some manual calculations, could be useful as well.

[dubvr6]

Hi,

I have come across the following formula and want to verify the validity of the formula. I have a VM which is 140GB in size, dedupe 1.0x, compression 2.1x, the full back up is 1, increments is 14. The rest is default. Please can someone plugin the details in the formula for backup size. I am not sure what to put in the C and D.

Backup size = C * (F*Data + R*D*Data)
Replica size = Data + C*R*D*Data

Data = sum of processed VMs size by the specific job (actually used, not provisioned)
C = average compression/dedupe ratio (depends on too many factors, compression and dedupe can be very high, but we use 50% - worst case)
F = number of full backups in retention policy (1, unless backup mode with periodic fulls is used)
R = number of rollbacks (or increments) according to retention policy (14 by default)
D = average amount of VM disk changes between cycles in percent (we use 10% right now, but will change it to 5% in v7/8 based on feedback... reportedly for most VMs it is just 1-2%, but active Exchange and SQL can be up to 10-20% due to transaction logs activity - so 5% seems to be good average)

Thanks

[nielsengelen]

Have a look at http://rps.dewin.me - this helps you calculate backup storage needed.

Basically for C go for 50% and D for 10% (as it says).

[Slairoth]

Hi there,

i want to use GFS for our existing backup-system with these settings:

Total backup size: 6.2TB
copy ever: 1 Day
Restore Points: 3
Weekly backup: 4
Monthly backup: 2
Quarterly backup: 0
Yearly backup: 2

I made the check with the RSP-simulator, showing me a total-size of 17,7 TB for a one-year growth?
For a 3-year cycle, the total size would grow for only 3,7 TB up to 21,4 TB.
Can this be true - i need someone for comfirmation of this size please.

[foggy]

Markus, not sure what settings you used for simulation, but keep in mind that each GFS restore point is a separate full backup file (6.2TB in your case). Even considering the fact that a single restore point can be labelled as weekly/monthly/yearly at the same time, you need space for at least 6 fulls within 1 year with your requirements (1 regular (current) full, 4 weeklies, one additional monthly full). Next year you will need to add only one yearly full plus provision for data growth, hence the difference is not that high.

[barrell]

Hi Guys

Is this thread still up-to-date? I was looking at the 'Restore Point Simulator' as I have to work out how much storage to specify for a new install and was curious to know if this is still valid or there is a new version?

ALSO
Looking at the documentation for 'Veeam backup 9.5 user guide for vsphere' there is a section that quotes:
- Optimal is the recommended compression level. It provides the best ratio between size of the backup file and time of the backup procedure.

I've searched the whole document (over a 1000 pages) to find out what "OPTIMAL" is and there is no mention of the ratio? Is there a setting in the 'Restore Point Simulator' I need to use or am I missing the point?

Regards
LB

[foggy]

Hi Lee, yes, you can use the space calculation tool discussed here, link above always refers to the latest available version. Typical data reduction for optimal compression level is at least 50%, so corresponds to the "Conservative" setting in this tool.

[barrell]

Many thanks for the speedy reply, much appreciated indeed.

Regards
LB

[barrell]

Veeam compression calculations

I need to get this correct to spec my NAS so a couple of questions answered would really help me out, very new to the storadge side of this so sorry if these are somwhat obviouse!

I have the following requirments;

For daily backup jobs ranging from 4/7/8 days I am using;
Forever Incrimental
50% Conservative setting
40% change rate

For weekly jobs I'm not sure how to aproch this, I want to use the same Conservative/change rate and my requirments are a 1week and 6week job.
I'm looking to save as much space as possible so will only require to roll back to previouse week if required. Is the following correct and is there a recomended best practice for weekly (bit of an open question I know).

[foggy]

Could you please clarify, would it be a daily or a weekly backup copy? In other words, do you need any daily restore points on target or just a couple of weeklies (and would you like to have them as full backups or incremental chain will work?)?

[barrell]

For weekly I do not need daily restore points and full backup I think maybe better?? So I don't think I need incremental chain as we can just roll back to 7 days prior.