Comprehensive data protection for all workloads
Gostev
SVP, Product Management
Posts: 26876
Liked: 4356 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: convert 1 full and 30 incrementals to 31 full?

Post by Gostev »

Hi, Anton. There a very slim chance that you can do this through internal data mover command line, unless the process includes some external logic from .NET parts of code. I will be able to give you exact answer next week, when I am in the office and can talk to the devs. Thanks!

foggy
Veeam Software
Posts: 19540
Liked: 1781 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: [MERGED] convert 1 full and 30 incrementals to 31 full?

Post by foggy »

AntonZ wrote:I have a set of files from regular backup job (1 vbk and 30 vib incrementals only). I can import them to Veeam and recover VMs from ANY time point (by the way, US support told me that it is impossible without vbm file as incrementals will be unreadable - not true!).
Correct, VBM file is not required to import the backup chain but considerably speeds up this process.

AntonZ
Influencer
Posts: 13
Liked: never
Joined: Dec 04, 2014 4:22 am
Full Name: Anton
Contact:

Re: Offsite Incremental to Synthetic Full Utility

Post by AntonZ »

Thanks, guys!
At this point I have several sets of such files (vbk+n*vib) and several not working GFS jobs (from which I can't recover anything from incremental points).
I'm running out of space next week (I have 62TB volume with Win2012r2 dedupe, which currently gives me 140TB of backups) and need to do smth very quickly. The idea was to upload everything to amazon, but to do this I need vbk fulls (easier to recover, no need to download the whole chain).
If I need to pay for additional service to help me convert incrementals to fulls (if there is no internal utility or way to do it with available tools), I'm ready to pay.

Gostev
SVP, Product Management
Posts: 26876
Liked: 4356 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Offsite Incremental to Synthetic Full Utility

Post by Gostev »

AntonZ wrote:If I need to pay for additional service to help me convert incrementals to fulls (if there is no internal utility or way to do it with available tools), I'm ready to pay.
We don't do custom development, so that's off the table unfortunately.

AntonZ
Influencer
Posts: 13
Liked: never
Joined: Dec 04, 2014 4:22 am
Full Name: Anton
Contact:

Re: Offsite Incremental to Synthetic Full Utility

Post by AntonZ »

Got it. While I'm waiting for an answer from devs, one more question:
is there any difference in backups created by regular job and by backup copy job?
Would it be possible to create a new GFS job, run it once, then delete created backups in repository and move my one full and 30 incrementals to that folder and then trick Veeam somehow (edit vbm file or sql tables; change time on the server to go back), so it'll take my old backups and will start to transform them according to GFS job policy (let's say with 30 recover points)? Sounds like super dirty trick, but who knows...

dellock6
Veeam Software
Posts: 5940
Liked: 1762 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Offsite Incremental to Synthetic Full Utility

Post by dellock6 »

not sure the "super trick" could work, but in terms of file format, both backup and backup copy job create a vbk file with the same format.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2020
Veeam VMCE #1

Shestakov
Expert
Posts: 7328
Liked: 773 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Offsite Incremental to Synthetic Full Utility

Post by Shestakov »

Anton,
What problem are you going to solve with the trick?
You can just start a new job with GFS retention and keep the chain you described with it`s own proper retention. Thanks.

AntonZ
Influencer
Posts: 13
Liked: never
Joined: Dec 04, 2014 4:22 am
Full Name: Anton
Contact:

Re: Offsite Incremental to Synthetic Full Utility

Post by AntonZ »

The problem I have is that I have a set of files left from one of my GFS job. Vbm file shows 0 bytes and there is no such job in Veeam. So I just have some fulls and many incrementals. I need to convert those incrementals into fulls.

Shestakov
Expert
Posts: 7328
Liked: 773 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Offsite Incremental to Synthetic Full Utility

Post by Shestakov »

Thanks for the answer.
I`m just curious how converting increments into fulls would help you to save the space?
As I wrote above you can just split your historical backups between 2 repositories if you have a lack of space.

AntonZ
Influencer
Posts: 13
Liked: never
Joined: Dec 04, 2014 4:22 am
Full Name: Anton
Contact:

Re: Offsite Incremental to Synthetic Full Utility

Post by AntonZ »

I'm going to offload data to amazon (I need roughly more than 100TB). Currently I have 13 fulls and 90 incrementals in that folder. I need to have all fulls, so I'll not have a headache in future when I need to recover something from amazon. Obviously, I can't move the whole chain with incrementals to the cloud as when I need to recover smth in the middle, I'll have to get the whole chain.

Shestakov
Expert
Posts: 7328
Liked: 773 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Offsite Incremental to Synthetic Full Utility

Post by Shestakov »

I see your point. Let`s wait for A.Gostev`s answer and see if developers can help with that.

AntonZ
Influencer
Posts: 13
Liked: never
Joined: Dec 04, 2014 4:22 am
Full Name: Anton
Contact:

Re: convert 1 full and 30 incrementals to 31 full?

Post by AntonZ »

Gostev wrote:Hi, Anton. There a very slim chance that you can do this through internal data mover command line, unless the process includes some external logic from .NET parts of code. I will be able to give you exact answer next week, when I am in the office and can talk to the devs. Thanks!
Hi Gostev!
anything on this?

cmaier
Enthusiast
Posts: 39
Liked: 4 times
Joined: Feb 24, 2014 4:01 pm
Full Name: Christian Maier
Contact:

[MERGED] Feature request: Transform previous backup chains l

Post by cmaier »

Hi,

We use incremental backups for all our jobs and transform the previous chains into rollback on weekend. But this is done job by job so when there are several jobs transforming their chains the others have to wait for start until the first have finished the transformation.

It lasts half a day until some jobs start with their backups in our environment. So it would be nice to finish the incrementals for all jobs first and transform the chains after that.

Thanks
Best regards
Christian

foggy
Veeam Software
Posts: 19540
Liked: 1781 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Feature request: Transform previous backup chains later

Post by foggy »

Christian, are you chaining you jobs so that the job started after the previous one has finished?

ChrisL
Enthusiast
Posts: 46
Liked: 6 times
Joined: Mar 21, 2011 12:04 pm
Full Name: Chris Leader
Contact:

[MERGED] Feature Request - ad-hoc Transform job

Post by ChrisL »

Hi all, don't know if this is even practical, but I occasionally find the need to re-run a job with a transform on a day that's not set to actually run a transform. Perhaps the previous transform failed, for example, but I don't want to wait until the next scheduled transform in a few days time to retry it since by this time the forward-chain is getting a bit long and the transform will take that much longer.

Wonder if it's possible, on the backup job right-click menu in B&R, where there's the option to 'Start' and 'Active Full' to add an option to 'Run with transform' or maybe just 'Transform now'.

Without this, if I want to run a transform job ad-hoc, we need to dig through the job settings to tick the current day in the Transform options, run the job and then remember to return to un-tick that day so that it doesn't go ahead and do it again the following week. All entirely doable, but just a bit messy.

Might be a silly idea, and obviously not expecting you to develop an option that I'm the only person in the world would use (!) but perhaps there's others out there in a similar position..

Shestakov
Expert
Posts: 7328
Liked: 773 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Feature Request - ad-hoc Transform job

Post by Shestakov »

Hi Chris!
Are we talking about synthetic full run?
Thanks for the feedback!

ChrisL
Enthusiast
Posts: 46
Liked: 6 times
Joined: Mar 21, 2011 12:04 pm
Full Name: Chris Leader
Contact:

Re: Feature Request - ad-hoc Transform job

Post by ChrisL »

Hi, yes that's right, sorry missed that bit out!

Perhaps I should rephrase that as an option to run an ad-hoc Synthetic Full now, in the same way as you can run an Active Full with a right-click. The transform will presumably then just follow automatically.

futureweb
Enthusiast
Posts: 67
Liked: 6 times
Joined: Sep 03, 2015 12:15 am
Full Name: Patrick
Contact:

Re: Offsite Incremental to Synthetic Full Utility

Post by futureweb »

For testing purposes (getting errors with synthetic full), it would be very helpful to force a synthetic full.

The problem is,if you have a full today, do incs today, the "synthetic full" is ignored for today, so to be able to test, I have to wait until tomorrow.
Thats why a force would be helpful.

veremin
Product Manager
Posts: 17882
Liked: 1673 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Offsite Incremental to Synthetic Full Utility

Post by veremin »

The feature request is noted.

If don't mind using PowerShell, take a look at the script provided in this thread.

Thanks.

futureweb
Enthusiast
Posts: 67
Liked: 6 times
Joined: Sep 03, 2015 12:15 am
Full Name: Patrick
Contact:

Re: Offsite Incremental to Synthetic Full Utility

Post by futureweb »

I read this thread already, but it doesn't do any different as if I set days in the GUI.
As it also won't do a synthetic full, if already a full was done on the same day (to note, I tried it with the powershell commands already, just to be sure).

For automatic tasking (as if you don't want weekly synthetic full, the script would work fine), not for testing if you search for an error. We go for weekly synthetic.

Thanks
Patrick

olafurh
Veeam Vanguard
Posts: 15
Liked: 15 times
Joined: Oct 29, 2014 9:41 am
Full Name: Olafur Helgi Haraldsson
Location: Iceland
Contact:

[MERGED] Feature request: Defer Merge

Post by olafurh »

Hi,

While running backups in full (veeam!) speed over the night, you are putting a lot of write load on your disk arrays, both in terms of IO and throughput. It would be smart, to have the option to defer the merge process on some jobs (the jobs could contain big BI analytic machines with terabytes of changed data every night for example) to another window than the backups are running in. It would put less load on both arrays and repositories servers when they should be busy writing down million dollar data and get a better utilization out your infrastructure investment in a less busy window.

ps, yes, or use refs and utilize fast clone...

olafurh

nmdange
Expert
Posts: 498
Liked: 127 times
Joined: Aug 20, 2015 9:30 pm
Contact:

Re: Feature request: Defer Merge

Post by nmdange »

The heavy IO on merge happens on the backup repository, not your production array. So if you're following best practices, that would be a completely separate device from your production environment. It would be very bad if your backups were comingled with your production data.

Poweruser
Expert
Posts: 211
Liked: 10 times
Joined: Jul 25, 2018 4:12 pm
Full Name: Poweruser
Contact:

Re: Feature request: Defer Merge

Post by Poweruser »

anyway the feature could be derived to another idea:
make inc. backups using high priority with max. resources.
and then the later process like merging could be background process low prio only.
so it wont eat up your resources if you plan to do a backup during worktime.

olafurh
Veeam Vanguard
Posts: 15
Liked: 15 times
Joined: Oct 29, 2014 9:41 am
Full Name: Olafur Helgi Haraldsson
Location: Iceland
Contact:

Re: Feature request: Defer Merge

Post by olafurh »

yes I know that the merge process dose not happen on the same arrays/disk boxes as the production vm data live :) But to ease up the load on the backup repository spindles/arrays would be nice by be able to defer the merge out side of a backup window.

Poweruser
Expert
Posts: 211
Liked: 10 times
Joined: Jul 25, 2018 4:12 pm
Full Name: Poweruser
Contact:

Re: Feature request: Defer Merge

Post by Poweruser »

so you can finish backups fast and rebuild reverse incrementals on standby/background

Reklov
Enthusiast
Posts: 81
Liked: 4 times
Joined: Aug 28, 2014 12:16 pm
Contact:

[MERGED] How to trigger a synthetic full

Post by Reklov »

Hi,
how can I start a synthetic full without or seperate from a Job ?

I' starting a Job by script and setup some resources into a specific "Backupmode". The Incrementaljob runs only 20 Minutes, so my resources can be moved back to non Backupmode, but this will be done at the end of the job ( defined a synthetic full every Saturday ) by a script. The synthetic full takes a lot of time. During this, the backupmode is not nessesary to may Resources ( they dont work in Backupmode ).

So I need to start the creation of a synthetic full separate.

Any possibility ?

Thanks,
Volker

foggy
Veeam Software
Posts: 19540
Liked: 1781 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Offsite Incremental to Synthetic Full Utility

Post by foggy »

Hi Volker, please review this thread for possible options. Thanks!

odge
Veeam ProPartner
Posts: 76
Liked: 7 times
Joined: Apr 14, 2011 3:20 pm
Full Name: Matthew Ogden
Contact:

Re: Offsite Incremental to Synthetic Full Utility

Post by odge »

+1

Reklov
Enthusiast
Posts: 81
Liked: 4 times
Joined: Aug 28, 2014 12:16 pm
Contact:

Re: Offsite Incremental to Synthetic Full Utility

Post by Reklov »

@foggy
thanks for this.
Unfortually I'nd found any Possibility to start only a synthetic Full, without starting the according Backupjob ( like in the PS Script ).
The only thing I want to do is to fire out a command to start a synthetic full, based on the incrementals without add a new incremental.

Any idea ?

foggy
Veeam Software
Posts: 19540
Liked: 1781 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Offsite Incremental to Synthetic Full Utility

Post by foggy »

This is currently not possible.

Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 31 guests