Comprehensive data protection for all workloads
joshuastenhouse
Novice
Posts: 8
Liked: never
Joined: Dec 02, 2011 10:18 am
Full Name: Joshua Stenhouse
Contact:

Huge v6 bug with job scheduling

Post by joshuastenhouse » Dec 02, 2011 10:34 am

I have found a major bug in v6 occurring in both an upgrade and a new install. My ticket ref is 5157348.

On Wednesday I upgraded my v5 Veeam server to v6. Immediatly all of my jobs started showing an incorrect next run time which didnt reflect what was actually set in the shcedule for the job. I thought this may of been a GUI issue, however my jobs then started running on the incorrect next run time. This included major VMs which shouldn't be run on a snapshot in business hours!

To rule out the upgrade as the issue I created a new VM and isntalled V6 from scratch using a local SQL express instance. I created a new job and scheduled it to only run on Wednesdays, the next run column will only show it to run on 03/12/11 rather than 07/12/11 no matter which day I pick that is not today or past Saturday. Thus ruling out any potential upgrade issue as its affecting a completely new install. My Veeam server, vcenter server and ESXi hosts are all showing the correct date and time and none have been altered recently.

I raised a fault at 13:30 on 30/11/12. It has taken until 10:10 02/12/12 for Veeam to acknolwedge they can replicate this fault in one of their test environments. I'm baffled as to how this hasn't been tested? Isn't backup scheduling a key part of backup software?

Gostev
SVP, Product Management
Posts: 24804
Liked: 3563 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Huge v6 bug with job scheduling

Post by Gostev » Dec 02, 2011 10:52 am

Hi Joshua, do you possibly have full backup scheduled to happen on Saturday?

joshuastenhouse
Novice
Posts: 8
Liked: never
Joined: Dec 02, 2011 10:18 am
Full Name: Joshua Stenhouse
Contact:

Re: Huge v6 bug with job scheduling

Post by joshuastenhouse » Dec 02, 2011 11:01 am

It is configured for reversed incremental so no day is specified advanced settings>backup>days tab.

On the active full backup option it is set to "weekly on selected days" and then every day is selected to ensure Veeam performs a full backup each time. If I un-tick this option completely the next run time is still incorrect.

Gostev
SVP, Product Management
Posts: 24804
Liked: 3563 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Huge v6 bug with job scheduling

Post by Gostev » Dec 02, 2011 11:19 am

Hi Joshua, please edit the job settings and under backup modes, switch to Incremental backup option, uncheck "Enable synthetic fulls", and then go back to reversed incremental option (the synthetic full schedule seem to wrongly affect this).

Also, a bit "cleaner" way to setup your desired schedule would be:
1. Use the incremental backup mode, uncheck Enable synthetic full.
3. Check Perform active full, Weekly, Wednesday.
3. On the job schedule step, set your job to run Daily, on these days, Wednesday.

Hope this helps.

joshuastenhouse
Novice
Posts: 8
Liked: never
Joined: Dec 02, 2011 10:18 am
Full Name: Joshua Stenhouse
Contact:

Re: Huge v6 bug with job scheduling

Post by joshuastenhouse » Dec 02, 2011 11:26 am

Haha, fantastic! So the bug is that if the sub check box enable synthetic full is checked then even if reversed incremental is selected then it overrides the scheduled date?

I have just tested this on my production server and it has resolved the issue. I just need to edit the other 38 jobs then and we are good to go.

46 hours since raising a ticket and no resolution. 1 hour since creating a forum post and its solved. Go figure. Thanks.

Gostev
SVP, Product Management
Posts: 24804
Liked: 3563 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Huge v6 bug with job scheduling

Post by Gostev » Dec 02, 2011 11:36 am

Usually it is faster through support, but they have been slammed past 2 days as it usually happens during major releases.

What helped here is I approached this differently - instead of troubleshooting your issue through the logs (which I cannot do anyway), I tried to create the backup schedule you need they way I would do it (as posted above). And I did not see the issue present there. Then, I deduced the difference down to that checkbox ;)

joshuastenhouse
Novice
Posts: 8
Liked: never
Joined: Dec 02, 2011 10:18 am
Full Name: Joshua Stenhouse
Contact:

Re: Huge v6 bug with job scheduling

Post by joshuastenhouse » Dec 02, 2011 11:53 am

Sorry, not working completely in production. I have a job scheduled to run every monday morning at 4:30am. I have unchecked the enable synthetic full box yet it is still saying next run date is 03/12/11 rather than 05/12/11?

Gostev
SVP, Product Management
Posts: 24804
Liked: 3563 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Huge v6 bug with job scheduling

Post by Gostev » Dec 02, 2011 12:02 pm

You should select only Wednesday in active full schedule (since this is what you want), and not every day. Otherwise, the active full backup will be created every single day, as instructed.

Full backup schedule now overrides main job schedule, this part is actually by design. Previously, people would often mistakenly schedule full backup for days when the job is not scheduled to run, and full backup would never create. There were support cases with people ending up with chains of hundreds of incremental backups without any fulls :D so now, full backup is always performed according to the schedule that is set for it (no matter if the job is scheduled to run on this specific day or not).

joshuastenhouse
Novice
Posts: 8
Liked: never
Joined: Dec 02, 2011 10:18 am
Full Name: Joshua Stenhouse
Contact:

Re: Huge v6 bug with job scheduling

Post by joshuastenhouse » Dec 02, 2011 12:14 pm

Back through the jobs I go! Thanks.

ThomasMc
Expert
Posts: 293
Liked: 17 times
Joined: Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell
Contact:

Re: Huge v6 bug with job scheduling

Post by ThomasMc » Dec 02, 2011 12:25 pm

joshuastenhouse wrote:Back through the jobs I go! Thanks.

Code: Select all

function Amend-SynJob ($jobName) {
	$job = Get-VBRJob -Name $jobName
	$holdingobj = $job | Get-VBRJobOptions
	$holdingobj.BackupTargetOptions.TransformFullToSyntethic = $false
	$holdingobj.BackupTargetOptions.TransformIncrementsToSyntethic = $false
	$job.SetOptions($holdingobj)
}

Code: Select all

Get-VBRJob | %{Amend-SynJob $_.Name}
This will untick Enable Synthetic Full & Transform for all jobs

Sethbartlett
Expert
Posts: 282
Liked: 25 times
Joined: Nov 10, 2010 6:51 pm
Full Name: Seth Bartlett
Contact:

Re: Huge v6 bug with job scheduling

Post by Sethbartlett » Dec 02, 2011 12:52 pm

You are everywhere Thomas! I feel like a second me has hit the forum with a "Why not use powershell?" method :P Nicely done.
Skype: Sethbartlett88 - Make sure to label who you are and why you want to add me ;)
Twitter: @sethbartlett
If my post was helpful, please like it. Sometimes twitter is quicker to hit me up if you need me.

Gostev
SVP, Product Management
Posts: 24804
Liked: 3563 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Huge v6 bug with job scheduling

Post by Gostev » Dec 02, 2011 12:58 pm

Seth, you sound quite jealous ;)

ThomasMc
Expert
Posts: 293
Liked: 17 times
Joined: Apr 13, 2011 12:45 pm
Full Name: Thomas McConnell
Contact:

Re: Huge v6 bug with job scheduling

Post by ThomasMc » Dec 02, 2011 1:42 pm

Gostev wrote:Seth, you sound quite jealous ;)
Seth is my Yoda :D


and thanks Seth :)

real_tarantoga
Enthusiast
Posts: 58
Liked: 7 times
Joined: Aug 20, 2009 12:32 pm
Location: Germany
Contact:

Re: Huge v6 bug with job scheduling

Post by real_tarantoga » Dec 05, 2011 12:25 pm

Thank you, Anton!
After upgrading this morning there weren't no more jobs scheduled to next sunday ... :roll:
unchecking the "enable synthetic fulls" box resolved the behaviour.

bug bug bug :wink:

Bye, Stefan

joshuastenhouse
Novice
Posts: 8
Liked: never
Joined: Dec 02, 2011 10:18 am
Full Name: Joshua Stenhouse
Contact:

v6 replication job scheduling issue

Post by joshuastenhouse » Dec 22, 2011 3:34 pm

[merged]

Hi,

I've created a new job, never run before with no seeding. I've set the schedule to run daily and selected only wednesdays at 7am. No backup window is configured and its not set to retry. This is a new v6 server, not an upgrade.

When I click finish and go back to the main screen, it says the next run date is 24/12/11. This is incorrect, it should be 28/12/11? i've tried removing the scheduling, disabling and re enabling the job to no avail. Please advise.

Gostev
SVP, Product Management
Posts: 24804
Liked: 3563 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Huge v6 bug with job scheduling

Post by Gostev » Dec 22, 2011 4:04 pm

Hi - this is the same issue, also described in more details in the sticky "Known Issues" topic. Thanks!

joshuastenhouse
Novice
Posts: 8
Liked: never
Joined: Dec 02, 2011 10:18 am
Full Name: Joshua Stenhouse
Contact:

Re: Huge v6 bug with job scheduling

Post by joshuastenhouse » Dec 22, 2011 4:28 pm

Ah yes, didn't see the extended list :-) Can you post a sample of this script rather than having to wait for support?

Gostev
SVP, Product Management
Posts: 24804
Liked: 3563 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Huge v6 bug with job scheduling

Post by Gostev » Dec 22, 2011 5:43 pm

Once the script gets sufficient validation, it will be included in the support KB article - until then, you can only get it from support.

micwe
Influencer
Posts: 18
Liked: never
Joined: Mar 20, 2011 8:56 am
Full Name: Mickeywe
Contact:

Veeam Backup V6 schedule Bug

Post by micwe » Dec 25, 2011 10:18 am

[merged]

In the last year I was a very happy Veeam Backup customer
Veeam Backup 4V 5V works grate for my ESXI 4.0 SMB environment

VMWARE vSphere5 new release is available for the last mounts
It seems Veeam has release V6 without enough QA software debug

Last week I upgrade my existing VMWARE environment to V5.0
And install new win2008 r2 for Veeam Backup & Replication 6.0.0.15
Unfortunately my experience with the last Veeam Build is bad

It started with this - Hot add is not supported for this disk - 6.0.0.153
Hot add is not supported for this disk - 6.0.0.153

Then this - v6 recurring VM copy not overwriting previous copies
v6 recurring VM copy not overwriting previous copies

And now new bug from this weekend - all my schedule jobs start automatically not in the correct date
I have set backup tasks - VM will be backup once during week days
But all schedule task start working yesterday one by one in time setting like schedule task but not correct date
I was very surprise to see seven tasks running last night that WAS NOT PLANNED

Is this a known issue and is there a fix for this?

Gostev
SVP, Product Management
Posts: 24804
Liked: 3563 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Huge v6 bug with job scheduling

Post by Gostev » Dec 25, 2011 10:41 am

Hi, see above for the scheduling bug.

Please note that hot add issue is not specific to Veeam v6, and moreover can be reproduced without Veeam B&R present in the environment (there is PowerShell script in the corresponding topic). There are 2 separate issues that can cause this, both are unrelated to issues with the Veeam code.

VM copy not overwriting previous copies was caused by the move to the new architecture with backup repositories (not a bug). However, based on a number of request we will make this behavior optional. With any software, I always recommend testing the new major release before upgrading to it - especially when there are big architecture changes.

If you'd like to discuss these 2 issues in more details, please feel free to post in the corresponding topics (not to derail this thread).

Thanks.

micwe
Influencer
Posts: 18
Liked: never
Joined: Mar 20, 2011 8:56 am
Full Name: Mickeywe
Contact:

Re: Huge v6 bug with job scheduling

Post by micwe » Dec 25, 2011 11:27 am

Gostev wrote:Hi - this is the same issue, also described in more details in the sticky "Known Issues" topic. Thanks!
can you please post a link for this page I can't find it

micwe
Influencer
Posts: 18
Liked: never
Joined: Mar 20, 2011 8:56 am
Full Name: Mickeywe
Contact:

Re: Huge v6 bug with job scheduling

Post by micwe » Dec 25, 2011 12:00 pm

o.k found it
[BEFORE YOU UPGRADE] Summary of known v6 issues

I am using a VM copy task and replication task
Sorry but there is no "Full Synthetic" option under this tasks
This option is only avalible on backup task

Gostev
SVP, Product Management
Posts: 24804
Liked: 3563 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Huge v6 bug with job scheduling

Post by Gostev » Dec 25, 2011 6:31 pm

Please, read the topic that you found.

micwe
Influencer
Posts: 18
Liked: never
Joined: Mar 20, 2011 8:56 am
Full Name: Mickeywe
Contact:

Re: Huge v6 bug with job scheduling

Post by micwe » Dec 26, 2011 6:20 am

Gostev wrote:Hi Joshua, please edit the job settings and under backup modes, switch to Incremental backup option, uncheck "Enable synthetic fulls", and then go back to reversed incremental option (the synthetic full schedule seem to wrongly affect this).

Also, a bit "cleaner" way to setup your desired schedule would be:
1. Use the incremental backup mode, uncheck Enable synthetic full.
3. Check Perform active full, Weekly, Wednesday.
3. On the job schedule step, set your job to run Daily, on these days, Wednesday.

Hope this helps.
Gostev I have read your post several times and can't figure it out
Please bear in mind that I am using a copy vm and vm replication jobs
There is no "synthetic full" option under these jobs
I am doing a vm copy one a week so I don't understand why all job started to work on Saturday as well

Vitaliy S.
Product Manager
Posts: 23001
Liked: 1557 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Huge v6 bug with job scheduling

Post by Vitaliy S. » Dec 26, 2011 8:08 am

Hi Mickey, I guess Anton was referring to this line:
Gostev wrote: Some of these issues are being considered for the next v6 patch, I will update this topic once it is available.

micwe
Influencer
Posts: 18
Liked: never
Joined: Mar 20, 2011 8:56 am
Full Name: Mickeywe
Contact:

Re: Huge v6 bug with job scheduling

Post by micwe » Dec 26, 2011 8:19 am

Thanks for replying Vitaliy
Maybe Gostev can clarify this issue

Gostev
SVP, Product Management
Posts: 24804
Liked: 3563 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Huge v6 bug with job scheduling

Post by Gostev » Dec 26, 2011 11:53 am

Gostev wrote:Issue 9
Jobs scheduled to start on selected days only will always start on synthetic full day when Enable synthetic full checkbox is checked, even if incremental backup mode is not used. To workaround, uncheck this checkbox and save settings. If your replication jobs are also affected, our support can provide a script resolving this.

micwe
Influencer
Posts: 18
Liked: never
Joined: Mar 20, 2011 8:56 am
Full Name: Mickeywe
Contact:

Re: Huge v6 bug with job scheduling

Post by micwe » Dec 26, 2011 12:10 pm

So only Script can fix this on vm copy and replication tasks ?
Are you referring to the script post on first page in this post ?

Gostev
SVP, Product Management
Posts: 24804
Liked: 3563 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Huge v6 bug with job scheduling

Post by Gostev » Dec 26, 2011 12:17 pm

I am referring to the official script from our support.

micwe
Influencer
Posts: 18
Liked: never
Joined: Mar 20, 2011 8:56 am
Full Name: Mickeywe
Contact:

Re: Huge v6 bug with job scheduling

Post by micwe » Feb 13, 2012 1:13 pm

Hello
For Future readsers
I can confirm that this issue was fixed
After installing Patch 2 (build number 6.0.0.164)

Post Reply

Who is online

Users browsing this forum: mbell98 and 14 guests