No Catalyst Copy support?

Availability for the Always-On Enterprise

Re: No Catalyst Copy support?

Veeam Logoby SyNtAxx » Thu Jan 14, 2016 3:33 pm

nefes wrote:Have you tried backup copy job in v9 between Catalyst stores already? With all improvements, made in v9, it should take reasonable time.

I haven't even installed the product yet. However, from past experience with Backup Copy jobs it involves creating more policies and creates more administrative effort and potentially more server horsepower (in my case not an issue) to allow for these jobs to remain running in the background. You haven't answered my question, or is your answer the only 'reliable' method is Veeam backup copies?

I've been looking for the information Gostev mentioned was presented at VeeamOn, is there availability to this data or a recording of the event ?

-Nick
SyNtAxx
Expert
 
Posts: 127
Liked: 14 times
Joined: Fri Jan 02, 2015 7:12 pm

Re: No Catalyst Copy support?

Veeam Logoby nefes » Thu Jan 14, 2016 3:43 pm

Since I'm not in Product Management team, I can't comment on any feature that will or will not be in any future version.
For v9 I am not aware of any other reliable way to copy backups from Catalyst to Catalyst except Backup Copy job.
nefes
Veeam Software
 
Posts: 552
Liked: 129 times
Joined: Mon Dec 10, 2012 8:44 am
Full Name: Nikita Efes

Re: No Catalyst Copy support?

Veeam Logoby Gostev » Fri Jan 15, 2016 11:29 am 1 person likes this post

Unfortunately, VeeamON sessions are available for attendees only.

Also, I just wanted to clarify the issue, as it is being misinterpreted elsewhere. Storage-based replication by itself is extremely reliable and efficient engine, which is why we ARE looking for ways to leverage it in our solution. I have not seen it corrupt "good" backups while replicating those, or anything like that.

What makes ANY storage-based replication a bad choice for data protection TODAY are the following major issues, which are not specific to HP StoreOnce:

1. It is not content-aware, and as such it will replicate bad (corrupted) data just as well as it replicates good data. So, if your source backup files are corrupted, they will still be replicated without raising any flags, leaving you with no ability to restore from either on-site or off-site repository in the time of need. This is not the case with Backup Copy jobs, which validates data as it copies it over.

2. It does not make you meet the 3-2-1 backup rule, that every data protection strategy must be designed around, in the "2" part. Specifically, there are no 2 different medias with this solution: 2 identical storage device in sync with each other cannot be consider separate medias by any means, as backups remain in the same fault. For example, if backups (or data inside of them) is deleted on the source storage (accidentally, or due to a bug in software) - then this deletion will be propagated to the target storage, leaving you with no good backups anywhere. Again, this is not the case with Backup Copy jobs. Along the same lines, Cryptolocker also became a big problem lately, with encrypted source backups being extremely reliably and efficiently replicated to the target storage ;)

There are also seems to be storage-specific replication issues with certain vendors, and I gave some recent examples during my VeeamON sessions. But I rather not post them here, as I know this will cause endless email threads with each vendor I will mention. And in any case, eventually they all come down to the above point 1 being the main issue.

Bottom line - thanks to our extremely large user base, we have been observing the above two issues consistently causing many of our users to actually lose their data, which resulted in my current position regarding leveraging "dumb" storage-based replication in data protection designs. But again, we do have plans to work closely with backup storage vendors to alleviate these issues by adding some intelligence on top of storage-based replication, which is indeed the ultimate off-siting engine.
Gostev
Veeam Software
 
Posts: 21622
Liked: 2411 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: No Catalyst Copy support?

Veeam Logoby pirx » Sat Jan 16, 2016 3:01 pm

We'll have soon 2x StoreOnce 6500, mainly because of the replication feature. So Catalyst Replication is must have for our next backup application. It' wouldn't make sense to buy this device (especially with the need to replicate data) and don't use it. I think there are also a lot of checks in Catalyst replication. Besides the 1500 VMs we've also to backup physical servers, so if we use this feature for them, there is no point to say it's not safe. We are also thinking about using the Oracle RMAN and MSSQL Catalyst Plugins (looking fine so far), also with replication.

I was really excited to hear that Veeam now supports Catalyst, but honestly you should have implemented Replication too and let the user choose what is best for them.
pirx
Enthusiast
 
Posts: 69
Liked: 7 times
Joined: Sun Dec 20, 2015 6:24 pm

Re: No Catalyst Copy support?

Veeam Logoby chjones » Sun Jan 17, 2016 7:34 pm

Wow this topic has really had some interest, wasn't expecting this. I also think this has gone down another path of Storage Replication (Catalyst Level) vs Application Replication (Veeam Level). Doesn't Veeam perform integrity checks on the backed up blocks and also has SureBackups, so wouldn't these capabilities negate the issue of replicating corrupt data? (Happy to be shot down on this one if I'm off the mark)

One issue I have with the way Veeam works with Catalyst is that, as I've previously stated, we use HP Data Protector to also backup physical servers to Catalyst Stores as well as initiate the replications. HP have a weird licence structure with StoreOnce. There is a StoreOnce Catalyst licence which is required on the StoreOnce to allow the use of Catalyst Stores. There is also a Catalyst Replication licence, which allows you to setup replication of Catalyst Stores between StoreOnce Devices.

We currently have the Catalyst Licence so we can write to StoreOnce Catalyst Stores with both Veeam and Data Protector.

The odd thing is that for Data Protector or Symantec NetBackup to write to a Catalyst Store all you need is the base Catalyst licence. You can then trigger a Catalyst Replication from within the backup product which will replicate the data to another Catalyst Store. Even though, at least in the Data Protector world, you need to have servers acting as StoreOnce Gateways at both the source and destination these gateways just act as listeners to each StoreOnce and report the replication progress back to the management server (Data Protector). The actual replication is completed solely by the StoreOnce devices, without the need for a Catalyst Replication licence.

However, if you to write data to a Catalyst Store and then have the StoreOnce automatically replicate new data to another Catalyst Store then you need the Catalyst Replication licence. This is not ideal as there is no knowledge or reporting that the replication has been completed, it's internally on the StoreOnce's. I don't really want to have to pay for another licence which for something which, I believe, is not as good as a controlled and reported replication.

I know in my company we would much rather have some level of awareness that a replication of data has occurred and be alerted when it fails.

My initial issue I posted about is that there are now several products that can write data to a StoreOnce Catalyst Store. There is Data Protector, NetBackup, BackupExec (I believe?), Veeam, MSSQL and Oracle, yet none of them can be made aware of data that each other has written. I don't need Data Protector or NetBackup to be aware of the contents of the data the other application has written to the Catalyst Store but I would love to be able to copy that data elsewhere, to another Catalyst Store or to an LTO tape. For example Data Protector doesn't need to know how to read the contents of a VBK file, that's Veeam's job, but would be such an improvement if Veeam data was written to a StoreOnce in a way that the other products could interact with the data and move it. That's my dream.
chjones
Enthusiast
 
Posts: 83
Liked: 25 times
Joined: Tue Oct 30, 2012 7:53 pm
Full Name: Chris Jones

Re: No Catalyst Copy support?

Veeam Logoby pirx » Tue Jan 19, 2016 7:25 am

Gostev wrote:1. It is not content-aware, and as such it will replicate bad (corrupted) data just as well as it replicates good data. So, if your source backup files are corrupted, they will still be replicated without raising any flags, leaving you with no ability to restore from either on-site or off-site repository in the time of need. This is not the case with Backup Copy jobs, which validates data as it copies it over.


But shouldn't Veeam notice that the source data is corrupt? I mean... the problem doesn't seem to be that the replication is failing and silently changing blocks. It's the fact that the source backup is corrupt. I haven't looked in detail in Veeam's backup copy (I see that you write that it checks the data), but there is also the source repository which is the base for the copy to a second location.

Gostev wrote:2. It does not make you meet the 3-2-1 backup rule, that every data protection strategy must be designed around, in the "2" part. Specifically, there are no 2 different medias with this solution: 2 identical storage device in sync with each other cannot be consider separate medias by any means, as backups remain in the same fault. For example, if backups (or data inside of them) is deleted on the source storage (accidentally, or due to a bug in software) - then this deletion will be propagated to the target storage, leaving you with no good backups anywhere. Again, this is not the case with Backup Copy jobs. Along the same lines, Cryptolocker also became a big problem lately, with encrypted source backups being extremely reliably and efficiently replicated to the target storage ;)


HP takes a replicated backup job as 2 different media. I'm not quite sure that the statement about the deleted files on one side is true for StoreOnce Catalyst Replication (not sure what the difference between Catalyst Copy and Catalyst Replication is). At the moment we are using Data Protector with Copy Jobs (Replication) and data that is deleted on one side is not automatically deleted on the other side. This is also true for HP's Oracle SO RMAN Plugin that is using Catalyst Copy.

Maybe we are talking about different things. I'm not talking about Replication inside StoreOnce which simply transfers all data from a source to a target store. I'm talking about StoreOnce Replication that is controlled by an application, in our case HP Data Protector or the RMAN Plugin. I'd like to see the same feature in Veeam. At the end it's an question how we can get the data to a second location and StoreOnce Replication looks good enough to us, especially with physical servers, VM's and a large amount of TB each day.
pirx
Enthusiast
 
Posts: 69
Liked: 7 times
Joined: Sun Dec 20, 2015 6:24 pm

Re: No Catalyst Copy support?

Veeam Logoby matsholm » Tue Jan 19, 2016 8:32 am

I agree with both "CraigL2112" and "SyNtAxx" when it comes with the lack of intregration with Storeonce Catalyst Copy. Saying from a software company that hardware replication isn't good enough is just crap. I've been working with Storeonce (and other DD and NetBackup) appliances for so many years with many big customers and I must say that Veeams reply to why not implement storage based replication is very strange. If you're not trust HPE do have a reliable block-replication between Catalyst stores then why do you trust HPE to even be a backup repository? With your approach your even saying that all DR setups in the world that rely on mirroring or other storage based replication is worthless?
How should we handle our big customers that were really looking forward to have a good replication implemented, "nefes" reply sying that backup copy jobs between copy jobs in v9 should be reasonable is not good enough. What is implemented in v9 that would be so much better handling a backup copy job comparing with catalyst copy? Can Veeam do a backup copy from one store to another without rehydrating the data?
/Mats
matsholm
Influencer
 
Posts: 18
Liked: 1 time
Joined: Tue Jan 19, 2016 8:02 am
Full Name: Mats Holm

Re: No Catalyst Copy support?

Veeam Logoby foggy » Tue Jan 19, 2016 1:07 pm

matsholm wrote:Saying from a software company that hardware replication isn't good enough is just crap.

We're not saying it is not good enough, we're saying completely different thing:

Gostev wrote:Also, I just wanted to clarify the issue, as it is being misinterpreted elsewhere. Storage-based replication by itself is extremely reliable and efficient engine, which is why we ARE looking for ways to leverage it in our solution. I have not seen it corrupt "good" backups while replicating those, or anything like that.

What makes ANY storage-based replication a bad choice for data protection TODAY are the following major issues, which are not specific to HP StoreOnce:

1. It is not content-aware, and as such it will replicate bad (corrupted) data just as well as it replicates good data. So, if your source backup files are corrupted, they will still be replicated without raising any flags, leaving you with no ability to restore from either on-site or off-site repository in the time of need. This is not the case with Backup Copy jobs, which validates data as it copies it over.


matsholm wrote:Can Veeam do a backup copy from one store to another without rehydrating the data?

Currently data needs to be rehydrated and sent over to the gateway server prior to be written to the second store. However, overall performance of the v9 backup copy job writing to the Catalyst Store is much better than the one of the v8 job writing to the CIFS share and all its synthetic activity is performed locally, without data rehydration.
foggy
Veeam Software
 
Posts: 15303
Liked: 1133 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: No Catalyst Copy support?

Veeam Logoby pirx » Tue Jan 19, 2016 1:27 pm

foggy wrote:Currently data needs to be rehydrated and sent over to the gateway server prior to be written to the second store. However, overall performance of the v9 backup copy job writing to the Catalyst Store is much better than the one of the v8 job writing to the CIFS share and all its synthetic activity is performed locally, without data rehydration.

And this rehydration is one point we want to avoid in future. Did you compare the performance of Veeam backup copy with SO replication? I fear in larger environment it's just a question if you are able to replicate/copy at all.
pirx
Enthusiast
 
Posts: 69
Liked: 7 times
Joined: Sun Dec 20, 2015 6:24 pm

Re: No Catalyst Copy support?

Veeam Logoby matsholm » Thu Jan 21, 2016 12:56 pm

foggy I still don't buy your arguments for not using storage based replication. Primary storage are normally not content aware either and a lot of companies and organizations have their primary storage replicated/mirrored (on block level not cintent) without any problems although your right about replication bad blocks. But StoreOnce and other vendors have a CRC engines working in background to make sure that data that the appliacation writes to storage are exactly the data written to disk (and also CRC for replication), so all data is checked.
To me it sound like you didn't have the time (or will) to implement this as a copy solution, are you planning to have that in future or on other appliance vendors?
matsholm
Influencer
 
Posts: 18
Liked: 1 time
Joined: Tue Jan 19, 2016 8:02 am
Full Name: Mats Holm

Re: No Catalyst Copy support?

Veeam Logoby foggy » Thu Jan 21, 2016 3:35 pm

It's not about bad blocks and CRC checks, it's mostly about replicating corrupt data, for example, if the source repository was compromised after data was written to the storage (think Cryptolocker or whatever). In this case you're just copying unrecoverable backups bit by bit to the secondary location.

Yes, we do have such plans, as stated above:
Gostev wrote:Bottom line - thanks to our extremely large user base, we have been observing the above two issues consistently causing many of our users to actually lose their data, which resulted in my current position regarding leveraging "dumb" storage-based replication in data protection designs. But again, we do have plans to work closely with backup storage vendors to alleviate these issues by adding some intelligence on top of storage-based replication, which is indeed the ultimate off-siting engine.
foggy
Veeam Software
 
Posts: 15303
Liked: 1133 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: No Catalyst Copy support?

Veeam Logoby SyNtAxx » Thu Jan 21, 2016 5:08 pm

So, there is no way to replicate an HP Catalyst store outside of a backup application (Veeam,Data protector)? In other words the HP StoreOnce 6500 can not replicate its own native catalyst stores on its own?

-Nick
SyNtAxx
Expert
 
Posts: 127
Liked: 14 times
Joined: Fri Jan 02, 2015 7:12 pm

Re: No Catalyst Copy support?

Veeam Logoby Gostev » Thu Jan 21, 2016 9:00 pm

I think it will be best to confirm directly with HP. There could be a workaround, or some functionality like that coming soon. For example, I know for sure that Data Domain can replicate DDBoost stores on its own, outside of a backup application.
Gostev
Veeam Software
 
Posts: 21622
Liked: 2411 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: No Catalyst Copy support?

Veeam Logoby chjones » Fri Jan 22, 2016 4:31 am

I've asked that question of HP and the answer was no, there is no way to have data on a Catalyst Store automatically replicated to another StoreOnce. A StoreOnce can have a replication licence applied, but that only works for VTL and NAS storage, not for Catalyst. With Catalyst (if you have a Catalyst Replication Licence) the replication must be initiated by your backup application.

HP weren't very forthcoming on reasons why the is the case, so I can only assume its so that replicated data is known to your backup product (whats the point of replicating that data if your backup product doesn't know it's been replicated to the destination? Importing Catalyst Objects into your backup product can be done, but it's not simple) ... and possibly also so HP retain some market share as their backup product, HP Data Protector, integrates the best with Catalyst and wants to manage the replication. I hope I'm wrong and this control over the replication can be allowed to other applications to control.
chjones
Enthusiast
 
Posts: 83
Liked: 25 times
Joined: Tue Oct 30, 2012 7:53 pm
Full Name: Chris Jones

Re: No Catalyst Copy support?

Veeam Logoby pirx » Fri Jan 22, 2016 12:08 pm 1 person likes this post

I still think that Veeam should be able to use the similar functionality as HP Data Protector with Catalyst protocol. HP DP knows about the replicated data and there is nothing going on behind DP's back. If objects in the source catalyst store are lost, the replicated data is still in the store on the other device. And I think it would also be possible to compare data in both stores to get the similar behaviour that Veeam backup copy has.

Maybe I'm missing the point. I can understand that this was not implemented in v9, but not this whole discussion about how unsafe appliance based replication is. If blocks are simply synced between both devices - yes. But AFAIK with StoreOnce Catalyst protocol, replication is managed by the backup application. So it's only a question what the application is capable of.
pirx
Enthusiast
 
Posts: 69
Liked: 7 times
Joined: Sun Dec 20, 2015 6:24 pm

PreviousNext

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Bing [Bot] and 1 guest