Daily full to Glacier (without workarounds?)

[mark49808]

Let me explain my use case here.

I have application logs I need to persist for compliance reasons with immutability. Today those files get dumped to a server, and I back them up to WORM tapes with NetBackup today. And once backed up to tape they are deleted because I only need them for regulatory reasons (unlikely ever to be retrieved). Having them on source disk is not necessary.

I want to recreate this with image-based backups: Veeam Performance Tier -> S3 -> Glacier. Because tapes are a nightmare.

But it seems only certain job types can go to Glacier.

Backup files with GFS flags assigned
VeeamZIP backup files
Exported backup files
Orphaned backups
Backups created with Veeam Backup for Red Hat Virtualization
Backups exported by K10 policies

Seems at best I get weekly Fulls to Glacier with GFS flags. But if I'm running the backup and deleting the files every day, will that not "miss" data for long term persistence? I'll get the Sunday Full for the week to Glacier, then that's it. Nothing else to Glacier? What about Monday thru Saturday? I get in most workloads a weekly full to Glacier is fine, but im doing "Archiving" and the supported flow seems to be more focused on "Backups" where potentially missing data during the week is OK. For me its not.

I suppose i can do some type of daily VeeamZIP via Powershell, but that seems messy and not long-term supportable as adjusting job parameters and whatnot is a pain vs the console/gui. Also not sure how job failure reporting works, etc.

Or i make sure that files never get deleted before the Weekly GFS full. But that's also problematic in the real world when compliance/regulatory things are involved.

Any pointers here?

[PetrM]

Hello,

Basically, you can either transport GFS-full backups to the archive tier or as you said leverage some PowerShell scripts to produce daily VeeamZip or Exported backups. Also, not sure that I clearly understand what "make sure that files never get deleted before the Weekly GFS full." mean and why do you need to make sure that these files are not deleted? The incremental backups cannot be transferred to the archive tier.

Thanks!

[mark49808]

Exactly. Incremental (daily) backups wont make it to Glacier so i need to be sure files are always present on source disk through the next weekly GFS full as I cannot rely on incremental (daily backup) to make it there. In my scenario I would pretty much ignore that backups during the week are occurring as they cant make it to Glacier. That is concerning for a tape replacement.

Seems to be a bit of a limitation compared to tape. I wish Veeam would allow for daily full backups integrated into the product rather than a weekly GFS full. to make it to Glacier.

Otherwise i'm stuck doing something like

Get-VBRRestorePoint -name whatever | sort CreationTime -desc |select -first 1 | Export-VBRRestorePoint -RetentionPeriod In7Years -Reason "Export to Glacier"

But wont that lose the space savings of Delta Fulls in Archive tier? In the UI I get the option (which i do not check) to "Store archive backups as standalone fulls". Seems like doing the export would be exactly this, and storing as standalone fulls.

I suppose I don't understand something, why its so complicated to get some data to glacier with more granularity than a week, without resorting to "interesting" things or mandating the data live on source disk until the next weekly full.

[veremin]

I suppose I don't understand something, why it's so complicated to get some data to glacier with more granularity than a week without resorting to "interesting" things or mandating the data live on source disk until the next weekly full.

Archive Tier has been designed to serve as storage for rarely accessed long-term data. Thus it's used mainly for offloading GFS restore points.

But wont that lose the space savings of Delta Fulls in Archive tier?

No, it will not. As long as the said checkbox is enabled, the already present blocks will be re-used.

Thanks!