-
- Enthusiast
- Posts: 94
- Liked: 3 times
- Joined: Jun 10, 2010 6:32 pm
- Contact:
Re: VMware CBT bug KB 2090639
How about some help for those of us not good with Powershell?
I run the 32 bit vmware powershell, and I try and copy each line from the KB into the command box. The first and 2nd line work , but not the middle section. Putting them all on one line doesn't work either.
I run the 32 bit vmware powershell, and I try and copy each line from the KB into the command box. The first and 2nd line work , but not the middle section. Putting them all on one line doesn't work either.
-
- Expert
- Posts: 117
- Liked: 31 times
- Joined: Oct 30, 2012 7:53 pm
- Full Name: Chris Jones
- Contact:
Re: VMware CBT bug KB 2090639
You need to run the scripts like you would a batch file or a vbscript, Powershell is no different. Copy all of the code into notepad and save it as a .ps1 file. Then in powershell you run the script. For example if you saved the file as C:\CBT-Fix.ps1, in the powershell prompt you would navigate to C:\ and run the command "./CBT-Fix.ps1" and it will execute all of the code (if you want to run a script you must put "./" preceding the file name)
-
- Influencer
- Posts: 12
- Liked: 1 time
- Joined: Dec 01, 2011 5:04 pm
- Full Name: Stephen
- Contact:
Re: VMware CBT bug KB 2090639
Per the Sunday's email from GOSTEV Veeam v8 has a work around for this bug?
does this mean if i upgrade i do not need to disable and enable the CBT on a VM with this problem?
Stephen
does this mean if i upgrade i do not need to disable and enable the CBT on a VM with this problem?
Stephen
-
- Expert
- Posts: 117
- Liked: 31 times
- Joined: Oct 30, 2012 7:53 pm
- Full Name: Chris Jones
- Contact:
Re: VMware CBT bug KB 2090639
The fix in v8 only works for any VMDKs that are increased in size AFTER v8 is installed. Any VMs already in your environment that have had their sizes changed in the past may be affected. You still need to run the CBT fix for existing VMs. The fix in v8 will only reset CBT automatically for you from that point forward.
Better to be safe than sorry is my motto when it comes to backups!
Better to be safe than sorry is my motto when it comes to backups!
-
- Novice
- Posts: 8
- Liked: 2 times
- Joined: Jul 20, 2009 7:05 pm
- Full Name: Kim Alexander Hansen
- Contact:
Re: VMware CBT bug KB 2090639
Maybe it is time for Veeam to add a option to periodically do a "full verify" on backups - which ignores CBT? This kind of thing would have been discovered and caught much ealier.. and give everybody some piece of mind? I understand the theory around surebackup, but even with this CBT bug, some OSes would boot and behave "normally".
-
- VeeaMVP
- Posts: 6165
- Liked: 1971 times
- Joined: Jul 26, 2009 3:39 pm
- Full Name: Luca Dell'Oca
- Location: Varese, Italy
- Contact:
Re: VMware CBT bug KB 2090639
Technically, you can already ignore CBT in a backup job and use disk scan.
But the problem here is not only ignoring CBT, because at next execution the bug will hit anyway, you have to properly disable and re-enable it in order to remove corrupted informations.
But the problem here is not only ignoring CBT, because at next execution the bug will hit anyway, you have to properly disable and re-enable it in order to remove corrupted informations.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software
@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Principal EMEA Cloud Architect @ Veeam Software
@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
-
- Novice
- Posts: 8
- Liked: 2 times
- Joined: Jul 20, 2009 7:05 pm
- Full Name: Kim Alexander Hansen
- Contact:
Re: VMware CBT bug KB 2090639
Luca - if that was a reply to me, I don't fully buy that argument. If the backup used CBT information but the verify did not use any CBT information. It would definately show that verification of the backup failed - every time you did a verify. It would be a indication that something is wrong? I think this bug whould have been caught years ago...
In any case, it is just a suggestion and something I have missed somewhat to be able to do. If the verify feature was "periodic", you could for example run a verify once a week.
In any case, it is just a suggestion and something I have missed somewhat to be able to do. If the verify feature was "periodic", you could for example run a verify once a week.
-
- VeeaMVP
- Posts: 6165
- Liked: 1971 times
- Joined: Jul 26, 2009 3:39 pm
- Full Name: Luca Dell'Oca
- Location: Varese, Italy
- Contact:
Re: VMware CBT bug KB 2090639
Everyone trusted completely CBT before the bug came out, however we did suggested for a long time to run surebackup at least for the most critical VMs exactly for this reason. Even without the CBT bug, different issues can come and they only way to catch any possible problem is to effectively test a restore, and surebackup is designed for this. Sorry if it seems a marketing answer, but the whole reason for designing surebackup is exactly this, and in a way the CBT bug is the biggest validation so far for this feature.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software
@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Principal EMEA Cloud Architect @ Veeam Software
@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
-
- Novice
- Posts: 8
- Liked: 2 times
- Joined: Jul 20, 2009 7:05 pm
- Full Name: Kim Alexander Hansen
- Contact:
Re: VMware CBT bug KB 2090639
The point here is that surebackup restore is not a guarantee that that the data is fully uncorrputed. You may be able to boot even with this CBT bug. So ideally you should have both. When it comes to CBT reliability, there surely are ways for the CBT to get corrupted. For example SAN crash / power failures etc.. If I'm the only one thinking it would be a good idea/feature to have - but I can live with that
-
- Influencer
- Posts: 12
- Liked: never
- Joined: Aug 06, 2014 8:13 am
- Full Name: A
- Contact:
Re: VMware CBT bug KB 2090639
I want to avoid snapping all my VMs at once, in batches or during biz hours... Can i do the following?
Disable CBT via Powershell on all VMs
Let the next Veeam job run (incremental)
Veeam will then enable CBT and snap/thaw
After this is done, i can upgrade to v8 and sleep soundly again?
Disable CBT via Powershell on all VMs
Let the next Veeam job run (incremental)
Veeam will then enable CBT and snap/thaw
After this is done, i can upgrade to v8 and sleep soundly again?
-
- Enthusiast
- Posts: 47
- Liked: 6 times
- Joined: Mar 06, 2012 11:45 pm
- Full Name: Nicolas Reutemann
- Contact:
Re: VMware CBT bug KB 2090639
I run the script on "biz hours", no impact, even on SQL servers. (I fully trust on Gostev words)
Before I run the script for all the VMs, I try over some VMs less critical, to see the impact and, after 15 trys I dont see any trouble, so, I run the script over all the infraestructure.
I dont run the incremental, I wait until the friday (when we run Incremental + Synthetic), run the script, upgrade to v8 and force an Active Full. The complete run took more time, but Im pretty sure, now, I got a good set of backups files.
Before I run the script for all the VMs, I try over some VMs less critical, to see the impact and, after 15 trys I dont see any trouble, so, I run the script over all the infraestructure.
I dont run the incremental, I wait until the friday (when we run Incremental + Synthetic), run the script, upgrade to v8 and force an Active Full. The complete run took more time, but Im pretty sure, now, I got a good set of backups files.
-
- Influencer
- Posts: 12
- Liked: never
- Joined: Aug 06, 2014 8:13 am
- Full Name: A
- Contact:
Re: VMware CBT bug KB 2090639
I definitely cant run snaps during biz hours, VMs are disconnected for a few seconds, unfortunately in my business sector, this is unacceptable.
-
- VeeaMVP
- Posts: 6165
- Liked: 1971 times
- Joined: Jul 26, 2009 3:39 pm
- Full Name: Luca Dell'Oca
- Location: Varese, Italy
- Contact:
Re: VMware CBT bug KB 2090639
Kim, I'm not saying you do not have a point here.
But as of now, CBT bug has shown up only when vmdk size is changed and crosses one 128GB boundary. So, also our workaround in V8 is more then what's technically needed to be safe. A complete disk scan, in my personal opinion (so not necessary the overall Veeam position on this) is an overkill, and the CBT reset upon disk size change is enough. And it should be applied now regardless a VM has just crossed the boundary, because if you have any VM above that size you don't know when it did changed the size or if it was deployed from the beginning above the boundary.
I'm more scared about old retention points, those kept for archival purposes, there's indeed a rick some of its content is unusable, thus making the whole idea of archiving useless...but at least we can fix the actual restore points.
PS: if you want to run a disk scan, technically you can already disable at all CBT in the backup job and let Veeam scan all blocks.
But as of now, CBT bug has shown up only when vmdk size is changed and crosses one 128GB boundary. So, also our workaround in V8 is more then what's technically needed to be safe. A complete disk scan, in my personal opinion (so not necessary the overall Veeam position on this) is an overkill, and the CBT reset upon disk size change is enough. And it should be applied now regardless a VM has just crossed the boundary, because if you have any VM above that size you don't know when it did changed the size or if it was deployed from the beginning above the boundary.
I'm more scared about old retention points, those kept for archival purposes, there's indeed a rick some of its content is unusable, thus making the whole idea of archiving useless...but at least we can fix the actual restore points.
PS: if you want to run a disk scan, technically you can already disable at all CBT in the backup job and let Veeam scan all blocks.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software
@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Principal EMEA Cloud Architect @ Veeam Software
@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
-
- Enthusiast
- Posts: 47
- Liked: 6 times
- Joined: Mar 06, 2012 11:45 pm
- Full Name: Nicolas Reutemann
- Contact:
Re: VMware CBT bug KB 2090639
I dont have any disconnection. I leave a ping running on some random VMs, and only have 1 ping with 250ms when the commit succeed.v334m wrote:I definitely cant run snaps during biz hours, VMs are disconnected for a few seconds, unfortunately in my business sector, this is unacceptable.
But, maybe, my infrastructure its no big enough to have a big impact.
-
- Veteran
- Posts: 338
- Liked: 35 times
- Joined: Jan 20, 2012 2:36 pm
- Full Name: Christensen Farms
- Contact:
Re: VMware CBT bug KB 2090639
Can someone from VEEAM please post a Powershell script that will work with one specified VM at a time? I think that would help a lot of people out after reading through this post. A lot of us want to be cautious and try the script out on a single VM or a few test VMs even though the script provided on here earlier that does all VMs at once is said to be safe to run, if we could have a script that runs against one VM at a time that would be nice.
I think it's important that this script is provided and verified by someone from VEEAM and not a forum end user. While I appreciate the other end user input on here, something this critical I want to hear directly from VEEAM.
Also, once this is done, as I understand, there is no way to verify that our VMs are safe moving forward? We just have to make sure we run the script on every VM and that is our assurance?
I think it's important that this script is provided and verified by someone from VEEAM and not a forum end user. While I appreciate the other end user input on here, something this critical I want to hear directly from VEEAM.
Also, once this is done, as I understand, there is no way to verify that our VMs are safe moving forward? We just have to make sure we run the script on every VM and that is our assurance?
-
- Enthusiast
- Posts: 53
- Liked: 2 times
- Joined: Feb 10, 2011 7:27 pm
- Contact:
Re: VMware CBT bug KB 2090639
this script posted earlier in the thread does just that and worked great for me: http://forums.veeam.com/vmware-vsphere- ... ml#p123778cffit wrote:Can someone from VEEAM please post a Powershell script that will work with one specified VM at a time?
It's not "from Veeam" but it is straightforward
-
- Enthusiast
- Posts: 94
- Liked: 3 times
- Joined: Jun 10, 2010 6:32 pm
- Contact:
Re: VMware CBT bug KB 2090639
That worked like a charm. Thanks!chjones wrote:You need to run the scripts like you would a batch file or a vbscript, Powershell is no different. Copy all of the code into notepad and save it as a .ps1 file. Then in powershell you run the script. For example if you saved the file as C:\CBT-Fix.ps1, in the powershell prompt you would navigate to C:\ and run the command "./CBT-Fix.ps1" and it will execute all of the code (if you want to run a script you must put "./" preceding the file name)
-
- Veteran
- Posts: 338
- Liked: 35 times
- Joined: Jan 20, 2012 2:36 pm
- Full Name: Christensen Farms
- Contact:
Re: VMware CBT bug KB 2090639
Ok I did this and it appears to work. Is there a line I can run right after I've done this to verify if CBT is in fact disabled for sure? When I go in though the vSphere client and look at the properties it still states that the disk CBT is turned on. I know the value in the script applies to the whole VM, but I just want a way to verify that after this script is run that CBT is in fact disabled for sure. Thanks!mrt wrote: this script posted earlier in the thread does just that and worked great for me: http://forums.veeam.com/vmware-vsphere- ... ml#p123778
It's not "from Veeam" but it is straightforward
-
- Influencer
- Posts: 17
- Liked: 3 times
- Joined: Dec 08, 2009 10:24 pm
- Contact:
Re: VMware CBT bug KB 2090639
Code: Select all
get-vm your_vm_name | ?{$_.ExtensionData.Config.ChangeTrackingEnabled -eq $true}
-
- Novice
- Posts: 5
- Liked: 1 time
- Joined: Dec 27, 2013 5:46 am
- Full Name: Chia-hung Tseng
- Contact:
Re: VMware CBT bug KB 2090639
Dears,
I met a problem when running the backup job after using the script of KB1940.
The script was run normally, with the CTK disable and snapshot created/removed.
The job always failed at the first execution after CTK has been reset. The error messages were as following:
"Job has failed unexpectedly"
And the following INCR or manually active backup are completed successfully.
The task log shows there's something wrong at the first backup session after CBT reset.
Can someone give me some hints?
Regards,
Joe
I met a problem when running the backup job after using the script of KB1940.
The script was run normally, with the CTK disable and snapshot created/removed.
The job always failed at the first execution after CTK has been reset. The error messages were as following:
"Job has failed unexpectedly"
And the following INCR or manually active backup are completed successfully.
The task log shows there's something wrong at the first backup session after CBT reset.
Can someone give me some hints?
Regards,
Joe
-
- Product Manager
- Posts: 20384
- Liked: 2295 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: VMware CBT bug KB 2090639
Hi,
Kindly, avoid posting log snippets, as it's against our forum rules. Please, contact our support team directly for any technical issue encountered.
Thanks.
Kindly, avoid posting log snippets, as it's against our forum rules. Please, contact our support team directly for any technical issue encountered.
Thanks.
-
- Novice
- Posts: 5
- Liked: 1 time
- Joined: Dec 27, 2013 5:46 am
- Full Name: Chia-hung Tseng
- Contact:
Re: VMware CBT bug KB 2090639
apologize for my mistake.
Because I'm verifying the impact of this VMware CBT bug for our customer.
I'm not sure if I can open a support case with my evaluation key.
So that I would like ask someone experienced.
Maybe I'll open a support request later to see if what they can help me.
Thanks for your reminder.
Because I'm verifying the impact of this VMware CBT bug for our customer.
I'm not sure if I can open a support case with my evaluation key.
So that I would like ask someone experienced.
Maybe I'll open a support request later to see if what they can help me.
Thanks for your reminder.
-
- Product Manager
- Posts: 20384
- Liked: 2295 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: VMware CBT bug KB 2090639
According to our policy , customers even with evaluation license are eligible for support. So, feel free to contact them. Thanks.
-
- VP, Product Management
- Posts: 27364
- Liked: 2794 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: VMware CBT bug KB 2090639
Guys,
If you're using Veeam ONE or Veeam MP for SCOM, then here is a small trick that would allow you to detect ALL VMs that had their virtual disks extended and could be potentially affected by this CBT bug.
Veeam ONE:
1. Navigate to VMware configuration tracking report pack and select Infrastructure Changes by Object report
2. In the parameters specify "Virtual Machine" as a reporting object
3. In the object properties choose "VM disk size"
Veeam MP:
1. Select Configuration Tracking and Alert correlation report
2. Choose "group changes by property"
3. In the reporting scope select "VMware virtual machines" as an object class
4. In the parameters choose "VMware Virtual Machine: Disk Allocation GB" property
Please be aware that you can select any reporting period in these reports. Hope you will find this tip useful.
Thanks!
If you're using Veeam ONE or Veeam MP for SCOM, then here is a small trick that would allow you to detect ALL VMs that had their virtual disks extended and could be potentially affected by this CBT bug.
Veeam ONE:
1. Navigate to VMware configuration tracking report pack and select Infrastructure Changes by Object report
2. In the parameters specify "Virtual Machine" as a reporting object
3. In the object properties choose "VM disk size"
Veeam MP:
1. Select Configuration Tracking and Alert correlation report
2. Choose "group changes by property"
3. In the reporting scope select "VMware virtual machines" as an object class
4. In the parameters choose "VMware Virtual Machine: Disk Allocation GB" property
Please be aware that you can select any reporting period in these reports. Hope you will find this tip useful.
Thanks!
-
- Service Provider
- Posts: 268
- Liked: 18 times
- Joined: Jan 21, 2011 10:10 am
- Full Name: Eric Boucq
- Contact:
[MERGED] VMware CBT data corruption: hot fix installation
Hello,
Following Gostev's weekly post, could you confirm that the hotfix for B&R v7 patch#4 to fix the VMware CBT data corruption issue just consists in .DLL copy in the Veeam directory ?
Thank you.
Regards,
Eric
Following Gostev's weekly post, could you confirm that the hotfix for B&R v7 patch#4 to fix the VMware CBT data corruption issue just consists in .DLL copy in the Veeam directory ?
Thank you.
Regards,
Eric
-
- Veeam Software
- Posts: 723
- Liked: 185 times
- Joined: Jun 05, 2013 9:45 am
- Full Name: Johan Huttenga
- Contact:
Re: VMware CBT data corruption: hot fix installation
You should replace the .dll files and then restart all Veeam services.
-
- Service Provider
- Posts: 268
- Liked: 18 times
- Joined: Jan 21, 2011 10:10 am
- Full Name: Eric Boucq
- Contact:
-
- Veeam Software
- Posts: 723
- Liked: 185 times
- Joined: Jun 05, 2013 9:45 am
- Full Name: Johan Huttenga
- Contact:
Re: VMware CBT data corruption: hot fix installation
No worries . (I like keeping an old copy of .dll files, but it should be fine)
-
- Veeam ProPartner
- Posts: 561
- Liked: 103 times
- Joined: Dec 29, 2009 12:48 pm
- Full Name: Marco Novelli
- Location: Asti - Italy
- Contact:
Re: VMware CBT bug KB 2090639
Hi guys, one question about the script to reset CBT: I have an host with both production VM and some replicated VM from another host
Can I run the script against all VM or should exclude the replicated (target) VM?
Thanks
Marco
Can I run the script against all VM or should exclude the replicated (target) VM?
Thanks
Marco
-
- Veeam Software
- Posts: 21137
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: VMware CBT bug KB 2090639
You can reset CBT on all of the VMs. Even if you're backing up replica VMs, CBT still cannot be used for them.
Who is online
Users browsing this forum: No registered users and 27 guests