HP StoreOnce Best Practices

[CraigL2112]

Hello everyone -

We're in the process of implementing Veeam (v8 w/ the 02 update) and a pair of HP StoreOnce 4500's.

Just curious if anyone can offer some 'Best Practices' for the StoreOnce? I've seen some articles (and even guides) have completely different settings in the 'Storage' tab under Advanced settings. How we've been running for the past couple of weeks is with inline dedupe OFF, swap blocks excluded, Compression at None and the Storage Optimization at LAN target. Doing Forward Incrementals w/ active fulls one day per week.

Dedupe on the StoreOnce side is very good, and jobs get done quickly. No issues there!

We're seeing what appear to be random issues with the StoreOnce throwing 'Cancelled paused Replication data job...' messages, and I'm wondering if it could be related to an incorrect setting.

Any help anyone could provide would be appreciated. Thanks in advance!

-Craig

[Andreas Neufert]

Hi, the settings are correct and best practices as well. (v8)
Maybe you can reduce the datastream with compression dedup friedly a bit more, so you can reduce the data a bit.

Never ran into this error, please open a ticket for it and upload logs. Please post the ticket number here.

[CraigL2112]

Hi Andreas -

Thank you for the reply. I can try turning on the compression to 'Dedupe-friendly' on a few of the jobs and see what it does. Thank you for the tip!

The warnings we are seeing are on the StoreOnce side -- not Veeam. With that being said, the warnings are always thrown around the time the Veeam backups are running. Will keep an eye on it and open a ticket once I have more information.

-Craig

[Andreas Neufert]

Then it is better to open a support case at HP.

There is as well another option to write faster in the StoreOnce instead of using an CIFS share, you can setup a linux box, mount the NFS share of the StoreOnce to it and use the mount point as a Veeam Linux repository.

[CraigL2112]

Wow, that is FASTER than using the CIFS share?

This begs another question.. how much faster will things run when there is native Catalyst support, especially if CoFC (Catalyst over Fiber Channel) is supported.

-Craig

[dellock6]

This is something we cannot answer since there's no support for Catalyst in our software; being comparable to EMC Boost, surely there could be a significant improvement.

[Andreas Neufert]

The main reason for Catalyst or DDDboost support is that you can work with syntetic processes to avoid Active Fulls and Snapshot commit problems because of it.

These Dedup Devices are very slow with random read streams and syntetic full processes will take forever and are not usable therefore. DDBoost or Catalyst give use the possibility to use Syntatic Fulls.
Performance is sometimes faster sometimes slower with DDBoost or Catalyst as we do not write 100% sequential.... but as I said you can avoid Active Fulls which is the main reason to use it.

As Luca said, we have DDBoost Support for DataDomain only in v8. We support as well FC connection to Datadomain as well.

@Craig, yes, NFS is in most cases way faster than CIFS.

[sundown78]

Hi Lucy, Andy,

is hp storeonce catalyst-support planned this year for B&R v8 or later v9/v10...waiting for it since ddboost is available

regards
René

[Andreas Neufert]

This is a question for our product management.

[foggy]

Definitely not for v8. However, I cannot comment on a more precise ETA.

[Vitaliy S.]

is hp storeonce catalyst-support planned this year for B&R v8 or later v9/v10

Now, you can review the details of the upcoming integration in this blog > HP StoreOnce Catalyst integration coming in v9!

[schenkewitz]

The announced StoreOnce Catalyst Integration in v9 is one of the most features we are waiting for.

Actually we have two 10TB HP StoreOnce VSA running and our first hardware-based StoreOnce 2900 with 24 TB is allready ordered togehter with an HP 3PAR 8200 2N with 16x 480GB cMLC SSDs

With v9 we will be able to use Veeam Storage Snapshots, StoreOnce RMC and StoreOnce Catalyst - this will be awesome.

[dellock6]

Just to clarify, RMC is not going to be supported in Veeam Backup & Replication v9.

[jbraes]

Hopefully this will improve StoreOnce performance.
We have a StoreOnce 4700 and performance is simply something non-existant.
Most Copy jobs take 12+ hours simply to complete (injecting incremental into full takes forever)
At this point I would not recommend this expensive dedup appliance, you better invest in DAS attached to your backup proxy.

[Andreas Neufert]

Catalyst Integration will enable us to do synthetic processing and merges on the StoreOnce.

Copy Jobs without catalyst integration can not run well, as they need to re hydrate and rewrite data and it take forever as you said.
Without Catalyst, best practices is to write with a standard Backup Job including Active Full+Incremental chain (dedup friendly compression and disabled Veeam Inline dedup).

[jbraes]

Andreas,

Please explain how DO i get my GFS rotation then.

[dellock6]

I guess first of all Andreas was meaning synthetic fulls and not active fulls.
For gfs, you want them in a primary job? Because it's not a feature actually available.
You can have gfs retention by leveraging backup copy jobs.

[foggy]

I guess Jeroen means the fact that "injecting incremental into full takes forever", so using backup copy jobs is not an applicable option at all. Catalyst integration improves transformation performance up to 10 times in our tests, so should address this pain to some degree.

[schenkewitz]

Catalyst Integration will enable us to do synthetic processing and merges on the StoreOnce.

Copy Jobs without catalyst integration can not run well, as they need to re hydrate and rewrite data and it take forever as you said.
Without Catalyst, best practices is to write with a standard Backup Job including Active Full+Incremental chain (dedup friendly compression and disabled Veeam Inline dedup).

Thats exactly our backup configuration and we're getting transfer rates between to 250 and 900MB per second with our Veeam B&R Jobs. And our dedupe ratio on the StoreOnce is at 24.22 : 1 now.

16,5 TB used space at 400 TB User Data written

This is amazing! We <3 our StoreOnce. HP told us dedupe ratio with around 20 : 1, but with every new week of backups our ratio is growing.

[andriktr]

Are there any recommendations for StoreOnce 4500 catalyst repository maximum concurrent tasks?

[CraigL2112]

@andriktr :

Are you seeing issues? We have 4500s as well and so far, at least for Copy Jobs (still not pleased Catalyst Copy is not supported) we seem to be stuck at 3. Any more than that and we get the -1404 errors.....

-Craig

[Gostev]

The main reason here is max amount of open files limit on StoreOnce. This is model specific and goes from 64 for the lowest end model to 256 or so for the highest end.

[CraigL2112]

If this is the case, it is another great reason why Catalyst Copy should be supported!

Further, we've just got a single job going right now with, like.. 20 VMs attempting to be copied and we're still randomly getting the -1404s (yes, I have an open case). Given we have multiple devices using StoreOnce as a backup target, it's feasible that it will be impossible for us to completely cut over to Catalyst stores/Backup copy jobs with Veeam.

FWIW, our existing jobs (that simply write to a CIFS share on StoreOnce) work like a champ. Zero issues -- they just keep working......

-Craig

[aevans@ricoh.com.au]

Hi,

Has Veeam or HP put together any whitepapers on HP StoreOnce and Veeam architecture/configuration recommendations post v9?

We are looking at StoreOnce as a possible replacement for current backup storage but still unsure whether we still need a non-dedupe landing zone for best restore performance.
Now that Veeam integrates directly is this still necessary? If so can part of the StoreOnce storage be set as non-dedupe to use as said landing zone or do you need a completely separate storage solution?

The full backup restore of a 2TB VM we did recently took over 24hrs. This was from an old IBM DS3500. Obviously upgrading this equipment we don't want to setup a dedupe appliance that takes this long or longer for restores.

I don't understand the point in purchasing hardware as a landing zone and then flushing to dedupe storage (atleast in our environment). We could just get a non-dedupe piece of hardware with additional capacity such as a bunch of Cisco UCS C3260's with 6TB NL-SAS drives.

Appreciate any thoughts or feedback StoreOnce users or Veeam team can provide.

Thanks.

[andriktr]

@andriktr :

Are you seeing issues? We have 4500s as well and so far, at least for Copy Jobs (still not pleased Catalyst Copy is not supported) we seem to be stuck at 3. Any more than that and we get the -1404 errors.....

-Craig

Hi, no we don't have any issues with Veeam v9 and HP StoreOnce catalyst stores, everything works very well also BC jobs performance much better than it was on StoreOnce CIFS shares. We using StoreOnce only for copy Jobs by default this option is 4. We have 6 copy jobs from different sites/repository's and this jobs are running at the same time that means if only 4 concurrent tasks are available only 4 VM's can be copied at the same time per all 6 jobs. So I Just want to find the optimal number for my environment.

[dellock6]

Hi,
Has Veeam or HP put together any whitepapers on HP StoreOnce and Veeam architecture/configuration recommendations post v9?

Andrew, we just published few weeks ago a new Configuration Guide, it may be worth a look:
https://www.veeam.com/wp-hpe-storeonce- ... guide.html

Luca

[aevans@ricoh.com.au]

Great thanks very much Luca I'll check that out.

[MatLoe]

Hi all,

Old post I know ... but we're in the process of installing a HPE StoreOnce 5100 and are wondering a few things such as:

- When adding a Catalyst backup repository freon Veeam Server, what'd be the best practice about "Limit the maximum concurrent tasks to: x" ?
I couldn't find anything regarding this setting, even in the pdf mentionned above. We use a 10GB ethernet connection.

- The "MTU question": we thought it'd be a good idea to use MTU = 9000 on both Veeam Proxy & StoreOnce but once configured it seems Veeam Server can't add a backup repository: we can only add it when using MTU = 1500. Any thought about this?

Thanks

Mat

[foggy]

Hi Mathieu, you can follow general recommendations regarding the number of concurrent tasks. Please see this thread for the answer to your second question. Thanks!

[MatLoe]

Thanks foggy (y)