Clarification about (compressed) block size and maintenance

[fika]

Hello,

Let's assume a backup job with storage set to "Local target".
Data deduplication activated.

According to the documentation, deduplication will then occur on 1024 KB blocks. Perfect.

Lets's then take the following 10 blocks .vbk file :
| A1 | B1 | C1 | D1 | E1 | F1 | G1 | H1 | I1 | J1 |
And the following 5 blocks .vib file :
| C2 | E2 | K2 | L2 | M2 |

In forever forward incremental mode, when .vib is merged to .vbk file :
- are old/expired blocks (C1, E1) removed ?
- is space then lost when old blocks are in the middle ?
- or are old blocks replaced by the new ones (C2, E2) ?
- are other new blocks added to the end of the .vbk file ?
- are all these operations made on a 1024 KB boundary, without touching the other existing .vbk blocks ?

When defragmenting and compacting full backup file :
- is this operation made on the 1024 KB boundary, just reordering the blocks, without changing their data, so that we finally found the exact same data in the .vbk file but sorted differently on the original 1024 KB boundary ?



Let's then assume compression is also enabled.

According to this, since compression ratio is very often around 2x, with this block size Veeam will write around 512 KB or less to the repository per each block.
As compression could give different results from one block to another, I am not sure I clearly understand.
Does this mean that every block will be written on the storage on a 512 KB boundary ?
Or can we then have blocks of different sizes in the same backup file ?

In other words, can we have a .vbk file with different block sizes ?
| _B_ | ___A___ | ___D___ | C |
Reorganized to the following ?
| ___A___ | _B_ | C | ___D___ |
Which would clearly be counterproductive for applications (sync program for example) which read the .vbk file.

When defragmenting and compacting full backup file :
- are blocks simply re-sorted, without creating new / recompressed blocks, so that we finally found the exact same data in the .vbk file but sorted differently ?
- on which boundary ? 512 KB ? "changing" boundary depending on each block size ? (related to my boundary question above)



Thank you very much for your support !

Fika

[foggy]

In forever forward incremental mode, when .vib is merged to .vbk file :
- are old/expired blocks (C1, E1) removed ?

They are marked as free to be then overwritten by the new ones.

- is space then lost when old blocks are in the middle ?

No, since they will be later overwritten by the new blocks.

- or are old blocks replaced by the new ones (C2, E2) ?

C2 will not necessarily replace C1, it can be written in the place of any block that is marked as free or appended to a file.

- are other new blocks added to the end of the .vbk file ?

Either that or written instead of existing blocks marked as free.

- are all these operations made on a 1024 KB boundary, without touching the other existing .vbk blocks ?

Correct.

When defragmenting and compacting full backup file :
- is this operation made on the 1024 KB boundary, just reordering the blocks, without changing their data, so that we finally found the exact same data in the .vbk file but sorted differently on the original 1024 KB boundary ?

During compact operation, new empty file is created and only required blocks are copied to it from the full backup file, without changing their contents or touching the boundaries.

As compression could give different results from one block to another, I am not sure I clearly understand.
Does this mean that every block will be written on the storage on a 512 KB boundary ?

No, in case block alignment is disabled in repository properties, blocks are written in a row, without any alignment. In case alignment is enabled,they are aligned at a 4KB boundary.

Or can we then have blocks of different sizes in the same backup file ?

Yes.

In other words, can we have a .vbk file with different block sizes ?
| _B_ | ___A___ | ___D___ | C |
Reorganized to the following ?
| ___A___ | _B_ | C | ___D___ |
Which would clearly be counterproductive for applications (sync program for example) which read the .vbk file.

Not sure I understand what you mean, please clarify.

When defragmenting and compacting full backup file :
- are blocks simply re-sorted, without creating new / recompressed blocks, so that we finally found the exact same data in the .vbk file but sorted differently ?

Blocks are not recompressed during compact operation.

- on which boundary ? 512 KB ? "changing" boundary depending on each block size ? (related to my boundary question above)

See above, depends on repository settings.