Google NearLine

[Kris]

Hiya all,

We are using Veeam B&R on VSphere5 in a very traditional way: BU2disk --> BU2disk --> BU2Tape.
As prices are dropping on Cloud Storage, this becomes more and more interesting.
For instance, Google Nearline (https://cloud.google.com/storage/pricin ... g-examples) is offering storage at $0.01/GB ($10.00/TB)

I was wondering if it is possible for Veeam B&R to offload backups to cloud storage such as Google Nearline (instead of tape), or maybe someone is already using this?

Thanks!
Kris

[Shestakov]

Hi Kris,
I believe Google NearLine is not among our partners so far. But there are lots of other cloud connect service providers. Check the link to find a provider in your country. Once you decided to go for a cloud, deployment scenario is explained here.
Thanks.

[Kris]

Hi Nikita,

Is a partnership with Veeam the only way to enable any BU2Cloud kind of solution?
Or are there other scenarios thinkable, like a CIFS, iSCSI or whatever kind of connection to a Cloud storage thingy?
(just brainstorming here )

Thanks,
Kris

[Shestakov]

Is a partnership with Veeam the only way to enable any BU2Cloud kind of solution?

Yes, because Service Providers(SP) need to run related services to supply the cloud connect. Aslo SPs support such features as WAN Acceleration, Instant provisioning etc.
Thanks.

[Vitaliy S.]

I was wondering if it is possible for Veeam B&R to offload backups to cloud storage such as Google Nearline (instead of tape), or maybe someone is already using this?

If you can expose this storage via CIFS or attach it to the Windows box, you will be able to use it as target storage for Veeam backups.

[Kris]

Thanks for the info.

I'm also looking at Amazon's Glacier, which is in the same ballpark and definately offers CIFS/iSCSI/NFS sorta connectivity.
It will be very interesting to see if and how this works out.

[dellock6]

Glacier per se doesn't offer any traditional connectivity, just like Google Nearline. But there are around tens of solutions to connect to those cloud storage solutions and expose them locally as a SMB/iscsi/NFS share. To name few, twinstrata, aws gateway, ctera, storsimple, steelstore....
What Nikita is listing to you are service providers running Veeam Cloud Connect, but this cannot be done on Google cloud.

Luca

[Meyercord]

Amazon Storage Gateway VTL (virtual tape library) might give you something like what you want. I don't believe Google has a comparable product yet, but I imagine it's coming soon.

The Storage Gateway VTL is an appliance you install on a VMware or HyperV host for $125/mo. You connect it to the Amazon S3 cloud, assign it some local cache storage (for low latency data) and some storage for an upload buffer and then it exposes a media changer and tape drives via iSCSI that you can attach through VEEAM. From the Amazon console, you then create virtual tapes which you can import into VEEAM and use just like normal tapes. The virtual library can hold 1500 tapes, and while the tapes are in the library, the data is stored in S3 at $0.03/GB/mo. If you export the tape from the library, it is moved to the "Virtual Tape Shelf", which transfers the data from S3 to Glacier, where it costs $0.01/GB/mo, the same as Google nearline.

Tapes in the Virtual tape Shelf at Glacier can later be imported back into the library as read-only tapes, but the download costs $0.09 per gigabyte and retrieval is slow. Also, because Veeam cannot perform direct restore from tape, restoring even a single file from a virtual tape would require retrieving all tapes in the backup set, writing all of the data to local disk and then performing the restore from disk. This means that you need a large enough cache to hold all of the tapes at once, plus additional disk storage to write it all back again.

I would consider VTL a good solution for highly redundant offsite storage of data that you plan to never access again, but are required by policy to retain. But its not especially cheap and if you ever do have to retrieve an archived tape from Glacier, it can potentially be very expensive and time consuming.

Amazon Announces Gateway-Virtual Tape Library (Gateway-VTL)
AWS Storage Gateway Pricing

[tommls]

Did Google (or ANYone) ever come out with anything to make Google cloud storage more 'compatible' with Veeam??

The most I can see so far is to copy a backup copy job's contents to Nearline or DRA, then bring it back down to local storage when/if necessary to restore therefrom.

Thank you, Tom

[mongie]

Tom, it can be slightly more automated than that...

I believe TwinStrata (and probably others) support Google Nearline, however there is a difference between Google Nearline + iSCSI Gateway and Veeam Cloud Connect.

With Nearline + iSCSI Gateway, your cloud storage appears as a normal Windows volume, and the gateway manages the translation from Block to Object storage. This is good in theory, but if you have a 1TB Backup file on your cloud storage and need to perform a file level recovery, you must download the whole backup file first (this is handled by the iSCSI Gateway, but is normally slow).

Veeam Cloud Connect is a less convenient, more expensive, channel based solution where the vendor can offer cloud storage (and in V9, replication and VM hosting I believe) along with some Veeam smarts in the cloud. This allows you to mount the backup files on the internet and only retrieve what you need. You can also use the WAN Optimisation feature to reduce the amount of traffic sent to the cloud provider.


Here is the calculation you have to do to find out whats best for you.

Google Nearline is 1c/GB. (Backblaze now offer cloud storage for $0.005c/GB). You also need a cloud gateway. The AWS gateway is $125/month, but it only works with AWS. There are 3rd party options available for other providers - EMC Twinstrata Cloud Gateway, NetApp AltaVault, etc.

Veeam Cloud Connect tends to be ~ 12c/GB+ (and sometimes there are per VM fees too)

If you have a very fast internet connection (actual speed required will depend on the size of your backup storage, but 100Mb+), the Google/AWS/Backblaze might work for you. It can certainly work, but you have to pay attention to the restore time. Veeam Cloud Connect will also work well, but can be 10x more expensive.

If you have a slow internet connection (Less than 100Mb), you could use Veeam Cloud Connect, but keep in mind your full restores are going to take time.



I hope this brain dump can help you.,

[Meyercord]

Veeam Cloud Connect is a less convenient, more expensive, channel based solution where the vendor can offer cloud storage (and in V9, replication and VM hosting I believe) along with some Veeam smarts in the cloud. This allows you to mount the backup files on the internet and only retrieve what you need. You can also use the WAN Optimisation feature to reduce the amount of traffic sent to the cloud provider.

Do you know if this can be "faked" with something like a cloud-hosted VM running a VEEAM proxy with WAN optimization and high-speed connection to backend cloud storage? Maybe something like an AWS EC2 instance with S3 storage buckets? Would this approximate the Cloud Connect features while allowing vendor-neutral storage, or is there some other benefit to the Cloud Connect service that would be missing from this arrangement?

[Gostev]

Veeam Cloud Connect is a less convenient

I would argue this part - as according to all polls, number one reason (and by far) for users to choose Veeam Cloud Connect over alternatives is specifically convenience and simplicity (due to the complete integration).

Overall, comparing Veeam Cloud Connect with "dumb" cloud storage will probably make a good VeeamON breakout session due to the amount of factors and features to be considered. You are listing a few VCC benefits, but there are definitely more to consider. For example, other solutions will not offer you periodic in-cloud backup health check with automated remediation (self healing), which is pretty important if you ask me - because what is the point in backups if you can't recover from them?

Even more importantly, the restore time... you have already mentioned the issue of having to download the entire backup file chain before the restore can be initiated with alternative solutions. With Veeam Cloud Connect, you can restore just what's needed directly from the cloud (single VM, single file and even an application item).

And there are many, many more considerations - both technical and procedural... I am not even attempting to have the detailed discussion on this topic over forum posts

Bottom line, looking at price alone is a wrong thing to do... if you are looking to save money - just don't do cloud backups at all. But if you decide to invest in cloud backups, you'd better make sure you are not throwing your money away for nothing (even if little money).

Nevertheless, it is a very useful brain dump, so thanks for taking time to post this!

[dellock6]

Well, the list of things you need to deploy yourself in a VM running in a cloud provider to make it a remote repository, by itself are already a lists of things you do not have to do with VCC. And that VM also needs to be managed once it's running (monitoring, patching, check for free space left and add additional disks to a point where a VM in the cloud usually cannot add more disks). This is comparable to having a server in a colocation, just in the fashion of a virtual machine. But still you have capex to pay for. With VCC, it's a pure pay-as-you-go.

Also, another important aspect that we always list is connectivity: you need to setup VPNs to reach VMs in the cloud, while with VCC you only need a straight TCP connection over the internet.

Finally, price is heavily dependant on the business plan of the provider offering Cloud Connect, but I've seen some providers able to offer really low prices per GB. And as Anton said, pure price per GB is not a fair comparison - even due to additional in-cloud compute requirement for VCC, which increases costs for a service provider.

[veremin]

Also, it's not a fair comparison, especially, when someone operates with the costs given by S3 or Glacier. While doing that, overwhelming majority of users tend to forget about the prices they will be charged with whenever the restoration need arises.

I'm not agitating for one solution or the other, just keep in mind that sometimes there is something else hiding behind low prices, such as download traffic prices or similar.

Thanks.

[tommls]

I think Nearline may be more useful for storing the results of backup COPY jobs aka 'archive' jobs than backups (fulls with incrementals), e.g. having the archives on local storage plus an offsite copy of the archives.
Then the file sizes and Internet speeds become the issue.
Thank you, Tom

[ITP-Stan]

Have you tried the Backup Copy Job "Merge" over a WAN connection without accelerator or Veaam Cloud Connect?
The merge takes a lot of time and a lot of bandwidth. I don't think it really is a viable solution except for very small backups.

[tommls]

Has anyone figured this out yet??
There's another thread on this topic with no conclusive answers.
Is it simply cheaper and better to use MS Azure since the cloud connect is included with the enterprise version and one can set it up without a consultant or 3rd-party company...
Thank you, Tom

[veremin]

As far as I'm concerned this type of storage can be used only in a combination with a 3-party software exposing it as ordinary system, such as NFS, ISCSI, CIFS, etc.

In future, kindly, don't create a separate thread for questions already discussed before, it's more convenient to have feedback regarding one particular functionality within one topic.

Thanks.

[primaryeng]

We are running Veeam and creating our backups to local storage and then pushing the VM images out to Google Nearline using Cloudberry (https://www.cloudberrylab.com/backup/wi ... erver.aspx) . Works well and provides the outcome we need. Just wish Veeam would partner up and make it a straight shot out.

[ITP-Stan]

In my opinion your best option is looking at a (Windows) cloud server with the needed amount of storage attached to it and connected with VPN (perhaps OpenVPN).
The cloud server would then act as a DATA mover for the repository you create, and merges would process on that server locally.
Depending on your licensed version you could also deploy WAN accelerators to speed up the DATA moving over slow WAN links.

Off-course when you choose a Cloud Connect repository everything is done for you without the need for setting up and managing.