Clarification about (compressed) block size and maintenance

Availability for the Always-On Enterprise

Clarification about (compressed) block size and maintenance

Veeam Logoby fika » Wed Sep 27, 2017 2:13 pm

Hello,

Let's assume a backup job with storage set to "Local target".
Data deduplication activated.

According to the documentation, deduplication will then occur on 1024 KB blocks. Perfect.

Lets's then take the following 10 blocks .vbk file :
| A1 | B1 | C1 | D1 | E1 | F1 | G1 | H1 | I1 | J1 |
And the following 5 blocks .vib file :
| C2 | E2 | K2 | L2 | M2 |

In forever forward incremental mode, when .vib is merged to .vbk file :
- are old/expired blocks (C1, E1) removed ?
- is space then lost when old blocks are in the middle ?
- or are old blocks replaced by the new ones (C2, E2) ?
- are other new blocks added to the end of the .vbk file ?
- are all these operations made on a 1024 KB boundary, without touching the other existing .vbk blocks ?

When defragmenting and compacting full backup file :
- is this operation made on the 1024 KB boundary, just reordering the blocks, without changing their data, so that we finally found the exact same data in the .vbk file but sorted differently on the original 1024 KB boundary ?



Let's then assume compression is also enabled.

According to this, since compression ratio is very often around 2x, with this block size Veeam will write around 512 KB or less to the repository per each block.
As compression could give different results from one block to another, I am not sure I clearly understand.
Does this mean that every block will be written on the storage on a 512 KB boundary ?
Or can we then have blocks of different sizes in the same backup file ?

In other words, can we have a .vbk file with different block sizes ?
| _B_ | ___A___ | ___D___ | C |
Reorganized to the following ?
| ___A___ | _B_ | C | ___D___ |
Which would clearly be counterproductive for applications (sync program for example) which read the .vbk file.

When defragmenting and compacting full backup file :
- are blocks simply re-sorted, without creating new / recompressed blocks, so that we finally found the exact same data in the .vbk file but sorted differently ?
- on which boundary ? 512 KB ? "changing" boundary depending on each block size ? (related to my boundary question above)



Thank you very much for your support !

Fika
fika
Influencer
 
Posts: 11
Liked: never
Joined: Wed Sep 20, 2017 8:10 am
Full Name: fika

Re: Clarification about (compressed) block size and maintena

Veeam Logoby foggy » Mon Oct 02, 2017 4:41 pm

fika wrote:In forever forward incremental mode, when .vib is merged to .vbk file :
- are old/expired blocks (C1, E1) removed ?

They are marked as free to be then overwritten by the new ones.

fika wrote:- is space then lost when old blocks are in the middle ?

No, since they will be later overwritten by the new blocks.

fika wrote:- or are old blocks replaced by the new ones (C2, E2) ?

C2 will not necessarily replace C1, it can be written in the place of any block that is marked as free or appended to a file.

fika wrote:- are other new blocks added to the end of the .vbk file ?

Either that or written instead of existing blocks marked as free.

fika wrote:- are all these operations made on a 1024 KB boundary, without touching the other existing .vbk blocks ?

Correct.

fika wrote:When defragmenting and compacting full backup file :
- is this operation made on the 1024 KB boundary, just reordering the blocks, without changing their data, so that we finally found the exact same data in the .vbk file but sorted differently on the original 1024 KB boundary ?

During compact operation, new empty file is created and only required blocks are copied to it from the full backup file, without changing their contents or touching the boundaries.

fika wrote:As compression could give different results from one block to another, I am not sure I clearly understand.
Does this mean that every block will be written on the storage on a 512 KB boundary ?

No, in case block alignment is disabled in repository properties, blocks are written in a row, without any alignment. In case alignment is enabled,they are aligned at a 4KB boundary.

fika wrote:Or can we then have blocks of different sizes in the same backup file ?

Yes.

fika wrote:In other words, can we have a .vbk file with different block sizes ?
| _B_ | ___A___ | ___D___ | C |
Reorganized to the following ?
| ___A___ | _B_ | C | ___D___ |
Which would clearly be counterproductive for applications (sync program for example) which read the .vbk file.

Not sure I understand what you mean, please clarify.

fika wrote:When defragmenting and compacting full backup file :
- are blocks simply re-sorted, without creating new / recompressed blocks, so that we finally found the exact same data in the .vbk file but sorted differently ?

Blocks are not recompressed during compact operation.

fika wrote:- on which boundary ? 512 KB ? "changing" boundary depending on each block size ? (related to my boundary question above)

See above, depends on repository settings.
foggy
Veeam Software
 
Posts: 15094
Liked: 1111 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson


Return to Veeam Backup & Replication



Who is online

Users browsing this forum: pstickle, Yahoo [Bot] and 39 guests