Comprehensive data protection for all workloads
mdornfeld
Expert
Posts: 125
Liked: 2 times
Joined: Mar 23, 2009 4:44 pm
Full Name: Matt
Contact:

Veeam Wishlist / Issues

Post by mdornfeld » Jun 16, 2009 2:03 pm

I think I came into Veeam expecting too much. Maybe Veeam sales just did a really good job, or maybe I just have unrealistic expectations for backing up my VM environment, probably a good mix of the two. Either way, I've been collecting a list of things that I would like to see improved, and most of them Veeam I'm sure is well aware of, but just in case, I wanted to list them all. As I've seen in the past, for some of them, Veeam will point directly at VMware and say, "there's nothing we can do, it's what we have to work with", and while that's understandable, I don't think that's the right approach. I guess my expectation as a customer is that Veeam will use their relationship with VMware to get these issues fixed on VMware's side as well as Veeam's side.

Backup & Restore
-Handle the restore of Virtual RDM's correctly
-Skip over reading blank space when doing backups
-Fix the issue where simultaneous backup jobs of highly transactional servers causes backups to fail.
-Removing a server from a Veeam backup job doesn't shrink the backup VBK file appropriately
Explaination: I removed a rather large server from our backup job, and the backup file stayed at 165 GB well past the number of rollbacks to keep. After I blew away the full backup, the next time it created a backup file, it was only 38 GB. In my mind, it should have auto-shrunk to the correct size on it's own.
-Increase speed of both backup and especially restoration.
-Become MS Exchange and MS SQL aware so it can take smarter/faster incrementals.

Scheduling
-Clean up scheduling engine (let me be able to specify exact multiple times per day that I want it to run a job - like windows scheduler)
-Enable me to do a grandfather, father, son rotation without scheduling 3 different jobs and having 3 different huge fulls hanging around.
-Enable the tape engine backup software to send/receive the following commands
1. Resume a Backup Job Schedule
2. Stop the scheduler for a particular backup job
3. See if a Veeam backup job is still in progress.

Other Issues
-The Veeam console acts flaky/crashes when opening it up in multiple sessions or closing it when a job is in progress.
-Veeam Support Logs - Everytime I seem to need the logs because a job failed/hung/whatever, they don't seem to be created in "Support Information" area. It's like they're deleted when the job fails or something.
-Stopping a Veeam job is like trying to stop a freight train. You can ask it to stop, but it's going to keep doing whatever it wants to do.

I hope Veeam takes this as constructive critisism. We obviously still chose to use Veeam over it's competitors, but we customers never focus on what's working right, do we :)

Gustov, is it possible to get a status on the aforementioned items in the list?

Thank you

Gostev
SVP, Product Management
Posts: 24017
Liked: 3254 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Wishlist / Issues

Post by Gostev » Jun 16, 2009 2:47 pm

Matt, thank you for your feedback. Yes, there is always place for improvement ;)

On the other hand, we've made something in 1.5 years that our competitors still do not have, having 1.0 version of their solutions released 3-4 years ago. So you can count on us - we will always be a few steps ahead. We have the best team in the world!

Good to hear our sales do good job :oops:

Yes, you may hear sometimes from support that "it is VMware problem", but they merely indicate this fact - it does not mean we are not planning on addressing this issue from our side. This is why I like these community forums - they give me visibility to those replies which make our customers not very happy. On the other hand, the problem we are seeing right now is VMware restricting all vendors to the use of public APIs only. And there is very little you can do to address VMware issues by using their own public APIs (which obviously all suffer from said issues too).

I've copied your bullet list below, see my answer in red.

Backup & Restore
-Handle the restore of Virtual RDM's correctly
If you are talking about produced VMDK naming issue, I believe it will be possible to resolve even with fix for current version. I can discuss this with our project lead when he returns from vacation.
-Skip over reading blank space when doing backups
For incremental backup: 4.0 (requires ESX 4.0 or later, will read changed blocks only).
For initial full backup: planned for 5.0

-Fix the issue where simultaneous backup jobs of highly transactional servers causes backups to fail.
Need more information on this one, but we did change some default freeze timeouts in 3.1 to address this issue.
-Removing a server from a Veeam backup job doesn't shrink the backup VBK file appropriately
Explaination: I removed a rather large server from our backup job, and the backup file stayed at 165 GB well past the number of rollbacks to keep. After I blew away the full backup, the next time it created a backup file, it was only 38 GB. In my mind, it should have auto-shrunk to the correct size on it's own.
4.0 via perform full backup command
-Increase speed of both backup and especially restoration.
Backup: significant improvements in 4.0 with ESX 4.0 hosts
Restore: planned for 4.5 via direct-to-SAN restores

-Become MS Exchange and MS SQL aware so it can take smarter/faster incrementals.
Not sure what you mean here. But again, 4.0 will have extremely fast incrementals with ESX 4.0 hosts (less than a minute).

Scheduling
-Clean up scheduling engine (let me be able to specify exact multiple times per day that I want it to run a job - like windows scheduler)
Not currently planned: based on feedback, such schedules are rarely used, and can be still implemented today by leveraging Windows scheduler and making it start Veeam Backup job from command line.
-Enable me to do a grandfather, father, son rotation without scheduling 3 different jobs and having 3 different huge fulls hanging around.
Please clarify?
-Enable the tape engine backup software to send/receive the following commands
1. Resume a Backup Job Schedule
2. Stop the scheduler for a particular backup job
3. See if a Veeam backup job is still in progress.
Can you clarify why this does not work for you? http://www.veeam.com/forums/viewtopic.php?p=5230#p5230
We are also considering native tape integration, and it is very high on our priorities list.


Other Issues
-The Veeam console acts flaky/crashes when opening it up in multiple sessions or closing it when a job is in progress.
Multi-user job management UI is a part of Veeam Backup 4.0
-Veeam Support Logs - Everytime I seem to need the logs because a job failed/hung/whatever, they don't seem to be created in "Support Information" area. It's like they're deleted when the job fails or something.
This issue with logs was never reported before. Logs are always created for each job, but they are never deleted in case of a failure of crash. If you can reproduce this behavior, we would be interested to investigate.
-Stopping a Veeam job is like trying to stop a freight train. You can ask it to stop, but it's going to keep doing whatever it wants to do.
Yes, it takes time to commit all VBK transactions, shutdows all agents and/or VCB processes, and especially to remove created snapshot. If we don't perform these operation and exit dirty, next job runs will still be failing, and there will be snapshot left - this would be very bad for us to do.

mdornfeld
Expert
Posts: 125
Liked: 2 times
Joined: Mar 23, 2009 4:44 pm
Full Name: Matt
Contact:

Re: Veeam Wishlist / Issues

Post by mdornfeld » Jun 16, 2009 5:21 pm

-Enable the tape engine backup software to send/receive the following commands
1. Resume a Backup Job Schedule
2. Stop the scheduler for a particular backup job
3. See if a Veeam backup job is still in progress.
Can you clarify why this does not work for you? http://www.veeam.com/forums/viewtopic.php?p=5230#p5230
Let me back up off this request, and make a much simpler request that would have more universal appeal. We're troubleshooting some issues right now, and the only way I have to stop a scheduled Veeam job from running is to go into the properties, write down the current schedule, reschedule it to not run, and then when I want to start it's regular schedule again, I have to go back in and reconfigure the whole schedule. Why isn't there "pause/unpause schedule" functionality, (much akin to Windows Scheduler "Enabled" check box).

Now it might be fair to ask me that if I like the functionality of windows scheduler so much, why don't I just use that? Here's the reason. I, personally, want a nice, integrated interface, that can tell me everything I want to know specifically about my VMware backup jobs in one place. This is a bit tongue-in-cheek, but If I didn't want that, I could have just as easily used a competitor's product. This is the same reason I don't want to use my tape engine to do all the scheduling (besides, my tape engine has enough problems running it's own backup jobs).

In all seriousness though, if I'm the only one who wants the ability to pause/unpause schedule ability, I'd say toss it out the window. But I guess my assumption is that other people would be interested in it as well. I'm sure you have ways (including these forums) of determining what your customers want.

Thanks as always for your help Gustov. I will try to clarify some other items when I have time.

Gostev
SVP, Product Management
Posts: 24017
Liked: 3254 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Wishlist / Issues

Post by Gostev » Jun 16, 2009 5:30 pm

Pause/unpause scheduled runs (internally, we call this functionality "temporary disable job") is actually in the list of low priority features for 4.0. Meaning, we are planning to add this feature if time allows to do it without slipping the release. So chances are good! Thank you.

mdornfeld
Expert
Posts: 125
Liked: 2 times
Joined: Mar 23, 2009 4:44 pm
Full Name: Matt
Contact:

Re: Veeam Wishlist / Issues

Post by mdornfeld » Jun 17, 2009 3:34 pm

-Become MS Exchange and MS SQL aware so it can take smarter/faster incrementals.
Not sure what you mean here. But again, 4.0 will have extremely fast incrementals with ESX 4.0 hosts (less than a minute).
Two things I mean here:
1. My *assumption* is that if Veeam sees an exchange or SQL database file change by one byte, it thinks it has to back up the entire file all over again, not just the changes that were made within that database file.
2. Have Veeam truncate trans logs for both SQL/Exchange.

mdornfeld
Expert
Posts: 125
Liked: 2 times
Joined: Mar 23, 2009 4:44 pm
Full Name: Matt
Contact:

Re: Veeam Wishlist / Issues

Post by mdornfeld » Jun 17, 2009 3:44 pm

-Enable me to do a grandfather, father, son rotation without scheduling 3 different jobs and having 3 different huge fulls hanging around.
Please clarify?
Example: Currently I can hang onto x amount of rollbacks. But lets say for instance I have a server backing up every 2 hours. I would like to retain the last 12 backups online (the last full day), then one backup per day for the last 7 days, and then one backup per a week for the last 4 weeks.

To do this currently with your software, I would have to have 3 jobs:
-2 hr backup job with 12 rollbacks
-Daily backup job with 7 rollbacks
-Weekly backup job with 4 rollbacks
This equates to 3 separate fulls.

Right now we accomplish this with just going to tape and using our tape rotation, so it's definately lower priority, but in the future it will become more useful as we phase out tape more and more.

mdornfeld
Expert
Posts: 125
Liked: 2 times
Joined: Mar 23, 2009 4:44 pm
Full Name: Matt
Contact:

Re: Veeam Wishlist / Issues

Post by mdornfeld » Jun 17, 2009 3:46 pm

We are also considering native tape integration, and it is very high on our priorities list.
Just my 2 cents. The company I work for would have very little current interest in this feature. We would definately like to see your developement efforts go towards getting the VMware backup to disk perfected.

lohelle
Service Provider
Posts: 77
Liked: 15 times
Joined: Jun 03, 2009 7:45 am
Full Name: Lars O Helle
Contact:

Re: Veeam Wishlist / Issues

Post by lohelle » Jun 17, 2009 6:14 pm

1.
I would like to be able to select VM's to be backed up in a job based on resource pools. Now we can select all VM's in a host and in a cluster (and individual), but not by resource pools / "folders".

Example:
Lets say we have 50 VM's in a Cluster. This cluster changing a lot. A lot of the VM's use local storage and others SAN storage. If we have two or more resource pools that separate these we could create 1 job for "local storage"-VM's and one for SAN storage. Without this we would need to update the job every time a VM is added to the cluster (or use service console backup for all VM's)

2.
When using service console backup:
Automaticly start one backup per host simultaneous (or x backups). This way we can create ONE job for all hosts, and still run many simultaneous backups at the same time. Simultaneous VCB backups could also be a nice feature, but then x jobs per SAN or target

3.
Failover to service console/agentless backup if VCB backup failes.

Gostev
SVP, Product Management
Posts: 24017
Liked: 3254 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Wishlist / Issues

Post by Gostev » Jun 17, 2009 7:37 pm

mdornfeld wrote:1. My *assumption* is that if Veeam sees an exchange or SQL database file change by one byte, it thinks it has to back up the entire file all over again, not just the changes that were made within that database file.
No, actually Veeam does incremental backup based on VMDK block level (1024KB block). So with your example, if Exchange or SQL database file changes by one byte, only 1024KB block involving this byte is transferred. If more than one byte changes, all blocks that got "dirty" due to this are getting transferred. For example, if you apply 4.5MB of new data, it makes 5 blocks dirty, in that case transfer "overhead" is just 0.5MB.
mdornfeld wrote:2. Have Veeam truncate trans logs for both SQL/Exchange.
If you are using Veeam VSS intergration, this should happen automatically after backup is done, as a part of regular Exchange VSS Writer activities (this is not a part of Veeam Backup functionality though).
mdornfeld wrote:Example... [skipped] ...Right now we accomplish this with just going to tape and using our tape rotation, so it's definately lower priority, but in the future it will become more useful as we phase out tape more and more.
Thanks for clarification, I understand. Definitely makes sense. I see it as a part of advanced scheduler we have been talking about above.
mdornfeld wrote:Just my 2 cents. The company I work for would have very little current interest in this native tape integration. We would definately like to see your developement efforts go towards getting the VMware backup to disk perfected.
In fact, these are very valuable 2 cents. :D
We are also having big internal debates about this specific feature.

Gostev
SVP, Product Management
Posts: 24017
Liked: 3254 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Wishlist / Issues

Post by Gostev » Jun 17, 2009 7:48 pm

lohelle wrote:I would like to be able to select VM's to be backed up in a job based on resource pools. Now we can select all VM's in a host and in a cluster (and individual), but not by resource pools / "folders".
I will check if we can add resource pools objects in selection in 4.0, this should be easy to do. But we already provide ability to create jobs by VM folders today. There is view mode selection above the UI control you use to pick the objects for backup.
lohelle wrote:When using service console backup: Automaticly start one backup per host simultaneous (or x backups). This way we can create ONE job for all hosts, and still run many simultaneous backups at the same time. Simultaneous VCB backups could also be a nice feature, but then x jobs per SAN or target
First, I wanted to note that at this time we did not have plans to enhance service console agent backup, because 2010 will be the last year for "fat" ESX. But we certainly have plans on implementing this functionality for remaining backup modes, and I see nothing preventing us from using this logic with network backup mode, since many restrictions do not really depend on backup mode (like ensuring that each LUN does not have more than one snapshot operation at a time, or preventing more than X jobs hitting the same target etc).
lohelle wrote:Failover to service console/agentless backup if VCB backup failes.
This is already implemented in the upcoming Veeam Backup 4.0, in vStorage API backup mode (VCB replacement).

lohelle
Service Provider
Posts: 77
Liked: 15 times
Joined: Jun 03, 2009 7:45 am
Full Name: Lars O Helle
Contact:

Re: Veeam Wishlist / Issues

Post by lohelle » Jun 17, 2009 7:57 pm

Gostev wrote: I will check if we can add resource pools objects in selection in 4.0, this should be easy to do. But we already provide ability to create jobs by VM folders today. There is view mode selection above the UI control you use to pick the objects for backup.
Ahh! I missed that one. Thank you!

me123
Influencer
Posts: 12
Liked: never
Joined: Apr 27, 2009 8:42 pm
Contact:

Re: Veeam Wishlist / Issues

Post by me123 » Jun 18, 2009 8:16 pm

This is already implemented in the upcoming Veeam Backup 4.0, in vStorage API backup mode (VCB replacement).
Does this mean that we will be able to have a SAN mode backup without having to use VCB?

Gostev
SVP, Product Management
Posts: 24017
Liked: 3254 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Wishlist / Issues

Post by Gostev » Jun 18, 2009 8:24 pm

That is correct, so no more VCB babysitting.

tsightler
VP, Product Management
Posts: 5294
Liked: 2146 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Veeam Wishlist / Issues

Post by tsightler » Jun 19, 2009 1:31 pm

I think my biggest wish would be the ability to set the number of simultaneous backups. We have about 40 VM's with about 7TB of data to be backed up. Since Veeam requires a full scan of every volume every night that means we pretty much have to read all 7TB of disk. Our typical backup speed is about 40-50MB/sec per backup, but we have about 500MB/sec available from the disk array. That's more than enough bandwidth to get the backups done in our overnight windows, but we have to run around 8-10 backups simultaneously to do it. That means trying to play with schedules to keep a reasonable number of jobs running. We end up using a mix of Network and VCB backups to get it done (network backups allow us to spread the load across many ESX servers and are great for "smallish" machines).

We considered simply grouping our backups into 8 or so jobs that are roughly the same size, but this leaves us with vbk files that are huge and, if there's a failure, the rollback can take and hour or more and the vbk is not usable for restores.

Actually, that's another big deal. A failed backup should not cause the vbk to be unusable. As it is now, what happens if disaster strikes during a backup. Your disk backups are no good and you'd likely be restoring your vbk's from your previous nights tape. I would be better if a new backup would write it's changes to a temporary file, and then commit the changes at the end. This would have a little more overhead, and wouldn't eliminate the possibility of a corrupt vbk (it could crash during the commit) but, in the more normal cases of a backup failing during it's processing, the recovery is just to delete the temp file, not to rollback transactions into the main file and hope for the best.

We came from vRanger and while we weren't very happy with it, mainly the pretty much useless file level restore, it was nice to be able to simply tell it what machines to backup, how many jobs to run simultaneously, and let it manage the details.

I realize this is probably going to be much less of an issue with vSphere 4 and the block change tracking, so maybe none of these issues are worth the effort, but I suspect many of us will be on ESX3.x for quite some time.

mdornfeld
Expert
Posts: 125
Liked: 2 times
Joined: Mar 23, 2009 4:44 pm
Full Name: Matt
Contact:

Re: Veeam Wishlist / Issues

Post by mdornfeld » Jun 19, 2009 1:37 pm

Actually, that's another big deal. A failed backup should not cause the vbk to be unusable. As it is now, what happens if disaster strikes during a backup. Your disk backups are no good and you'd likely be restoring your vbk's from your previous nights tape. I would be better if a new backup would write it's changes to a temporary file, and then commit the changes at the end. This would have a little more overhead, and wouldn't eliminate the possibility of a corrupt vbk (it could crash during the commit) but, in the more normal cases of a backup failing during it's processing, the recovery is just to delete the temp file, not to rollback transactions into the main file and hope for the best.
I wholeheartedly agree with the above statement. Is Veeam looking into this?

Locked

Who is online

Users browsing this forum: Exabot [Bot], Google [Bot], Karinne, salmanmo and 15 guests