Discussions related to using object storage as a backup target.
Post Reply
agrob
Veteran
Posts: 380
Liked: 48 times
Joined: Sep 05, 2011 1:31 pm
Full Name: Andre
Contact:

Question about Azure Blob Store - Tranfser Full Backup vbk

Post by agrob »

Good Day

I've got another Question about SOBR with Azure Blob Store Capacity Tier.
I did some tests with a backup job which protects one single Windows 7 VM. VMDK File ist about 40GB, 21GB are filled with data. The first SOBR Offload statistic did transfer about 21GB of DAta

19.02.2019 09:08:05 :: VM.vm-82100_3B51D2019-02-18T102556.vbk (21,0 GB) 36,6 GB read at 51 MB/s

This would be as expected.

Now i did start the backup job as active full. This created a new vbk file with about 21GB of Data.
Then i started a SOBR Offlaod. I would have expected so see another 21GB of data transfer, but it did only transfer about 2GB of Data
20.02.2019 12:28:26 :: VM.vm-82100_441ED2019-02-20T080023.vbk (21,5 GB) 2,0 GB read at 48 MB/s

Then i did another active full and started SOBR offlaod, this time it did transfer about 10GB of Data, not the 21GB as expected
20.02.2019 12:52:58 :: VM.vm-82100_A116D2019-02-20T110452.vbk (20,8 GB) 10,2 GB read at 54 MB/s


Where do those difference come from? Does it only transfer blocks which are changed, even if i run an active full? That would look like a sort of Deduplication. So in this case i have about 33GB (21 + 2 + 10) of total Storage needed in the azure blob store (in azure portal i see arround 27Gb), instead of about 63GB I would expect because 3 vbk a 21GB are offloaded.

Is that correct? So if i have weekly full backup of about 25TB and i offlaod 2 of those i dont need to transfer 50TB to Azure. First would be 25TB, second Full offlaod maybe only 5 TB (depends on change rate)?

Thanks
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Question about Azure Blob Store - Tranfser Full Backup vbk

Post by Gostev »

agrob wrote: Feb 20, 2019 12:00 pmWhere do those difference come from? Does it only transfer blocks which are changed, even if i run an active full? That would look like a sort of Deduplication.
Hello, yes that is correct. This is explained in the What's New document, please check it out as there are some other cool "hidden" optimizations, for example for restores. Thanks!
agrob
Veteran
Posts: 380
Liked: 48 times
Joined: Sep 05, 2011 1:31 pm
Full Name: Andre
Contact:

Re: Question about Azure Blob Store - Tranfser Full Backup vbk

Post by agrob »

Hi Gostev, thanks for your reply. I have to say i did not check the whats new pdf, shame on me... :(

I guess this is the section:
Space efficiency: In addition to source-side compression, backup files are offloaded to object storage in the forever incremental manner, thus preventing duplication between multiple full backups, which effectively provides source-side deduplication.
I guess this dedup is also done "only" within the same job as usual?
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Question about Azure Blob Store - Tranfser Full Backup vbk

Post by Gostev »

That is correct.
agrob
Veteran
Posts: 380
Liked: 48 times
Joined: Sep 05, 2011 1:31 pm
Full Name: Andre
Contact:

Re: Question about Azure Blob Store - Tranfser Full Backup vbk

Post by agrob »

thanks!
agrob
Veteran
Posts: 380
Liked: 48 times
Joined: Sep 05, 2011 1:31 pm
Full Name: Andre
Contact:

Re: Question about Azure Blob Store - Tranfser Full Backup vbk

Post by agrob »

want to reactivate this post. i have to calculate storage costs for long time protection onprem vs cloud storage. (i know the azure calculator and the other calculator from veeam)
lets assume we have total backup data of 30TB (Full Backup). Change rate is about 20% -> 6TB. I configure a Copy job to hold 12 monthly restore points

1. on a normal refs repostiory (on a NAS, DAS etc) i need 12*30TB = 360TB of Data, correct?
2. when offloading to azure blob, i need 30TB (first full) + 12 * 6TB (Change) = 102TB of Data, correct?

I can calculate the Price of this 102GB of Storage, but what about the read and write operations? i guess it depends on the block size veeam is moving the blocks to the blob store? what is the block size? is it 64kb over wan?

lets assume we have 64k per write or read, then i have to calculate this cost as well, right?
102TB = ~102'000'000'000KB / 64KB = 1’593’750’000 (needed writes)
then the write/read fee is per 10'000 i/o
1’593’750’000 / 10'000 = 159'375
10'000 Operations in the cool tier costs 0.1USD
159'375 * 0.1 = 15'938 USD

So we have 102TB Cool Tier = 1044USD/Month = 12'528 USD/year + 15'938 USD (to write all the data to the blob) = 28'466USD/Year

So i have to calculate arround 28500 USD for one Year of Retention points on azure blob cool tier if i get the above example right?
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Question about Azure Blob Store - Tranfser Full Backup vbk

Post by Gostev »

By default, Veeam uses 1MB blocks. So your read/write fee should be 16 times less than above... just under a thousand bucks. Thanks!
agrob
Veteran
Posts: 380
Liked: 48 times
Joined: Sep 05, 2011 1:31 pm
Full Name: Andre
Contact:

Re: Question about Azure Blob Store - Tranfser Full Backup vbk

Post by agrob »

Thanks Anton!
Post Reply

Who is online

Users browsing this forum: lperezg, ronnmartin61, sfirmes, YaCy [Bot] and 17 guests