Comprehensive data protection for all workloads
Post Reply
tfloor
Expert
Posts: 270
Liked: 14 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

2 questions about synthetic fulls

Post by tfloor »

Hello,

I'm using veeam 6 patch 3 on a physical windows 2008 server using san-mode with 4 jobs to backup vm's from the same template (linux / oes / win2003 / win2008)
I'm also writing to tape and have a good solution for that with backup exec that depends on datstamps of files.

Question 1: (see joblog)
What i see now is that for 1 job, that is set for 14 restore points there are some .VBK files. (ofcourse)
On friday i schedule the synthetic full. It's now sunday 27 Feb. So i had a new VBK file with a datestamp of 25 Feb.
That's correct. But i see that there are also 2 other previous vbk files that has a new datestamp of 25.

How is that possible?, is there a process that is changing datestamps of previous VBK files?, I only see this so far for 1 job.... Very strange.

24-2-2012 23:41:12 :: Job started at 24-2-2012 23:41:07
24-2-2012 23:41:12 :: Building VM list
24-2-2012 23:41:39 :: SRV4 is no longer processed by this job. Make sure this change is intentional.
24-2-2012 23:41:39 :: VM size: 1,3 TB
24-2-2012 23:41:39 :: Changed block tracking is enabled
24-2-2012 23:41:48 :: Preparing next VM for processing
25-2-2012 0:12:08 :: Processing 'SRV1'
25-2-2012 0:33:41 :: Preparing next VM for processing
25-2-2012 0:33:41 :: Processing 'SRV2'
25-2-2012 0:40:26 :: Preparing next VM for processing
25-2-2012 0:40:26 :: Processing 'SRV3'
25-2-2012 0:44:07 :: Preparing next VM for processing
25-2-2012 0:44:07 :: Processing 'SRV4'
25-2-2012 0:54:55 :: Preparing next VM for processing
25-2-2012 0:54:55 :: Processing 'SRV5'
25-2-2012 0:59:00 :: All VMs have been processed
25-2-2012 0:59:04 :: VM 'SRV4' is outdated and will be deleted
25-2-2012 0:59:21 :: VM 'SRV4' is outdated and will be deleted
25-2-2012 1:04:09 :: VM 'SRV4' is outdated and will be deleted
25-2-2012 1:07:51 :: Synthetic full backup created successfully
25-2-2012 16:33:49 :: Load: Source 9% > Proxy 33% > Network 3% > Target 98%
25-2-2012 16:33:49 :: Primary bottleneck: Target
25-2-2012 16:33:49 :: Job finished at 25-2-2012 16:33:49

I need to find a way that nothing is changing timestamps of files (otherwise the complete backupsolution to tape is useless)

Question 2: (see joblog)
The vm's backup on friday (synthetic full) is normally ready within 2 hours. But to create a VBK file the job finished about 15 hours later. Is that the normal process ? (or is this the time that veeam need to copy the full .VBK file to the target?)
So first it creates the incrementals (as usual), and then create a .VBK file (around 900GB) , that take up to 15 hours? (for about 5 vm's)

I hope someone can explain me this, so i can finalize my backup strategy.

Kind regards,

Tristan.

Gostev
SVP, Product Management
Posts: 26706
Liked: 4277 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 2 questions about synthetic fulls

Post by Gostev »

Hi,

1. This is not expected - I asked devs to check the code around this.

2. Yes, this is normal process. New synthetic full backup is created from the previous full backup, and incremental backups - so yes, in your case it is about taking 900 GB of data blocks from those files, and creating the new 900 GB file. The time it takes depends only on 2 things: the size of incremental backups to process, and performance of the backup storage. Looks like your backup storage is not very fast?

Thanks!

tsightler
VP, Product Management
Posts: 5679
Liked: 2499 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: 2 questions about synthetic fulls

Post by tsightler »

Gostev wrote:1. This is not expected - I asked devs to check the code around this.
I'm suspecting, based on the messages about "SRV4 is outdated", that there was a server named SRV4 deleted/removed from the job that hit the "Deleted VM Retention" period. When you hit this the blocks backing this VM are literally removed from the backup files which does cause the older files to be modified. Just a guess based on the log messages though.

Gostev
SVP, Product Management
Posts: 26706
Liked: 4277 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 2 questions about synthetic fulls

Post by Gostev »

This should not be happening in the forward incremental backup job - only backup modes with transform (those with a single full backup file) do this in v6.

tfloor
Expert
Posts: 270
Liked: 14 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

Re: 2 questions about synthetic fulls

Post by tfloor »

1. The Srv4 server was removed from inventory since it was orphaned. But then immediately added back to vcenter.
Because the vmid changed and veeam could not find the vm anymore I removed the server from veeam job and add again.
But it looks like i still get the warning messages in the reports about this vm
My vm retention period is 14 days. (the same as snapshots).

2. My target is a DD610 not very fast indeed. But I get a brand new enterprise class qnap this week. The highest model in range.

tfloor
Expert
Posts: 270
Liked: 14 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

Re: 2 questions about synthetic fulls

Post by tfloor »

tsightler wrote: I'm suspecting, based on the messages about "SRV4 is outdated", that there was a server named SRV4 deleted/removed from the job that hit the "Deleted VM Retention" period. When you hit this the blocks backing this VM are literally removed from the backup files which does cause the older files to by modified. Just a guess based on the log messages though.
That's correct. I had to re-add the server.
I still don't understand the vm retention period and what it does exactly.
As long as I have to backup to tape, I don't want modifications to older vbk files.

tfloor
Expert
Posts: 270
Liked: 14 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

Re: 2 questions about synthetic fulls

Post by tfloor »

It's possible that this was the case, but i'm using forward incrementals, and "GOSTEV" tells us, that this is not normal. For me i also don't want this, not only because of the change in datestamps, but i do not see any reason for us to change the files, because over about 2 weeks the files are deleted automatically. (14 snapshots)

So what is the correct answer for this.
- it's a bug (should not happen with forward-incr.)
- it's normal (maybe there is a way to disable the feature ......)

Thanks
Tristan

tsightler
VP, Product Management
Posts: 5679
Liked: 2499 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: 2 questions about synthetic fulls

Post by tsightler »

If Gostev says it should not happen, then I would say it must be a bug. I was only intending to indicating that I also see this behavior and describe the circumstances under which I have seen it occur.

tfloor
Expert
Posts: 270
Liked: 14 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

Re: 2 questions about synthetic fulls

Post by tfloor »

Gostev,

Can you confirm this is a bug? Or do you need more information. Maybe special log files.

Regards
Tristan

Gostev
SVP, Product Management
Posts: 26706
Liked: 4277 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 2 questions about synthetic fulls

Post by Gostev »

It should not happen, but it is not a bug. How's that :)

Seriously though - while I am still to confirm this, but from the above it looks like we are seeing left overs of valid v5 functionality which *should* have been removed from v6, but may be were not due to the lack of time or something. I was positive that they were in fact removed from v6, which is why I said "it should not be happening".

I will review the code with the devs (did not get a chance to do this yet), and if this is indeed the case, we will address that in the upcoming minor release. I will post here when I have definite information.

Thanks!

tfloor
Expert
Posts: 270
Liked: 14 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

Re: 2 questions about synthetic fulls

Post by tfloor »

Gostev,

That explains something. Wondering why this was the case in v5. I don't see it as a feature :)
Let me know if you have the results.

Regarding to the log file (maybe a bug) :
Why do i still see the this message each day :
24-2-2012 23:41:39 :: SRV4 is no longer processed by this job. Make sure this change is intentional.
While the server is backed up with status "success"

foggy
Veeam Software
Posts: 19465
Liked: 1767 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: 2 questions about synthetic fulls

Post by foggy »

Tristan, as you have stated above, you had to re-add the VM to the backup job. This message refers to the VM that was deleted from the job - just to point you that it is not backed up anymore. From the product's standpoint, the VM that was then added to the job is another VM, as VBR tracks VMs by their uniqie IDs (and the ID for this VM was changed after re-adding it to inventory).

tfloor
Expert
Posts: 270
Liked: 14 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

Re: 2 questions about synthetic fulls

Post by tfloor »

Foggy,
You are right. I've readd the server. And therefore it had the same name. That was confusing me. Underwater it's another vm according to the Id. So based on that story I think the message will go away after the vm retention period. I hope

Thanks
Tristan

Gostev
SVP, Product Management
Posts: 26706
Liked: 4277 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 2 questions about synthetic fulls

Post by Gostev »

Yep.

tfloor
Expert
Posts: 270
Liked: 14 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

Re: 2 questions about synthetic fulls

Post by tfloor »

Yep ? :)

Is that the definitive answer on question 1 or uhhh a yep only

Gostev
SVP, Product Management
Posts: 26706
Liked: 4277 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 2 questions about synthetic fulls

Post by Gostev »

The answer to "I think the message will go away after the vm retention period" :)

tfloor
Expert
Posts: 270
Liked: 14 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

Re: 2 questions about synthetic fulls

Post by tfloor »

That's clear.
i will wait and see if the message expired

tfloor
Expert
Posts: 270
Liked: 14 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

Re: 2 questions about synthetic fulls

Post by tfloor »

Gostev,

Any idea when I can receive the information you will get from dev?

I like veeam but i"m using it 2 weeks and have found 3 bugs :)
I hope there will come patches for that soon. I only have experiences with patch release times of other vendors.

Gostev
SVP, Product Management
Posts: 26706
Liked: 4277 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 2 questions about synthetic fulls

Post by Gostev »

Hi Tristan, reading through my response above the specific behavior being discussed in this topic is not a bug, but existing functionality of VM-level retention that we had in the product since v5, so this will definitely not get a patch. The current plan is to drop this functionality from the forward incremental backup mode in the next minor release. Thanks.

tfloor
Expert
Posts: 270
Liked: 14 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

Re: 2 questions about synthetic fulls

Post by tfloor »

Gostev,
So I can Not expect this in patch 4 or 5. I have to wait for 6.1 or something.
But what can I expect. Do I have to wait for weeks or months. Or half a year.
Is there maybe a way to disable the feature? Change the retention to 0 or 9999?

Gostev
SVP, Product Management
Posts: 26706
Liked: 4277 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 2 questions about synthetic fulls

Post by Gostev »

Good call. Changing the "Deleted VM retention" to a very high number for this job should effectively disable this functionality until 6.1 is released, which should definitely happen sooner than in 9999 days ;)

tfloor
Expert
Posts: 270
Liked: 14 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

Re: 2 questions about synthetic fulls

Post by tfloor »

Gostev,
I've set it to the max 999 days. So I hope to see the release within 3 years ;)
Thanks to confirm my own workaround haha
Thanks
Tristan

tfloor
Expert
Posts: 270
Liked: 14 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

Re: 2 questions about synthetic fulls

Post by tfloor »

It would be nice if the "known issues" list can be updated more frequently.
Fox example to add this one
And the schedule bug. And maybe the duplicate email notification,
This are only my own experiences, so there must be more that is known

Gostev
SVP, Product Management
Posts: 26706
Liked: 4277 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 2 questions about synthetic fulls

Post by Gostev »

Our support KB covers many more known issues and common questions. The known issue list on this forum is only for issues which have caused multiple support cases. I must keep it short and to the point - people don't read walls of text.

tfloor
Expert
Posts: 270
Liked: 14 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

Re: 2 questions about synthetic fulls

Post by tfloor »

I understand. So there is no place to find the known bugs. There is only a place to search for problems and get a result that could be a bug. ;)

Gostev
SVP, Product Management
Posts: 26706
Liked: 4277 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: 2 questions about synthetic fulls

Post by Gostev »

Correct. The best way is to search is the support KB with your issue/error.

ddockter
Enthusiast
Posts: 60
Liked: never
Joined: Sep 25, 2010 2:09 pm
Full Name: Doug Dockter
Contact:

Old .vib updated during full backup

Post by ddockter »

[merged]

My backups are scheduled to run fulls on Friday. I noticed this morning that one of the full backups also updated the time stamp of a .vib from 02-16. Why is that?

Post Reply

Who is online

Users browsing this forum: No registered users and 33 guests