Comprehensive data protection for all workloads
Post Reply
Kris
Novice
Posts: 3
Liked: never
Joined: Apr 15, 2015 8:51 am
Full Name: Kris ten Hoedt
Contact:

Google NearLine

Post by Kris »

Hiya all,

We are using Veeam B&R on VSphere5 in a very traditional way: BU2disk --> BU2disk --> BU2Tape.
As prices are dropping on Cloud Storage, this becomes more and more interesting.
For instance, Google Nearline (https://cloud.google.com/storage/pricin ... g-examples) is offering storage at $0.01/GB ($10.00/TB)

I was wondering if it is possible for Veeam B&R to offload backups to cloud storage such as Google Nearline (instead of tape), or maybe someone is already using this?

Thanks!
Kris
Shestakov
Veteran
Posts: 7328
Liked: 781 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Google NearLine

Post by Shestakov »

Hi Kris,
I believe Google NearLine is not among our partners so far. But there are lots of other cloud connect service providers. Check the link to find a provider in your country. Once you decided to go for a cloud, deployment scenario is explained here.
Thanks.
Kris
Novice
Posts: 3
Liked: never
Joined: Apr 15, 2015 8:51 am
Full Name: Kris ten Hoedt
Contact:

Re: Google NearLine

Post by Kris »

Hi Nikita,

Is a partnership with Veeam the only way to enable any BU2Cloud kind of solution?
Or are there other scenarios thinkable, like a CIFS, iSCSI or whatever kind of connection to a Cloud storage thingy?
(just brainstorming here :-) )

Thanks,
Kris
Shestakov
Veteran
Posts: 7328
Liked: 781 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Google NearLine

Post by Shestakov »

Kris wrote:Is a partnership with Veeam the only way to enable any BU2Cloud kind of solution?
Yes, because Service Providers(SP) need to run related services to supply the cloud connect. Aslo SPs support such features as WAN Acceleration, Instant provisioning etc.
Thanks.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Google NearLine

Post by Vitaliy S. »

Kris wrote:I was wondering if it is possible for Veeam B&R to offload backups to cloud storage such as Google Nearline (instead of tape), or maybe someone is already using this?
If you can expose this storage via CIFS or attach it to the Windows box, you will be able to use it as target storage for Veeam backups.
Kris
Novice
Posts: 3
Liked: never
Joined: Apr 15, 2015 8:51 am
Full Name: Kris ten Hoedt
Contact:

Re: Google NearLine

Post by Kris »

Thanks for the info.

I'm also looking at Amazon's Glacier, which is in the same ballpark and definately offers CIFS/iSCSI/NFS sorta connectivity.
It will be very interesting to see if and how this works out.
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Google NearLine

Post by dellock6 »

Glacier per se doesn't offer any traditional connectivity, just like Google Nearline. But there are around tens of solutions to connect to those cloud storage solutions and expose them locally as a SMB/iscsi/NFS share. To name few, twinstrata, aws gateway, ctera, storsimple, steelstore....
What Nikita is listing to you are service providers running Veeam Cloud Connect, but this cannot be done on Google cloud.

Luca
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Meyercord
Enthusiast
Posts: 36
Liked: 6 times
Joined: Jul 14, 2014 4:31 pm
Full Name: AJ Meyercord
Contact:

Re: Google NearLine

Post by Meyercord » 1 person likes this post

Amazon Storage Gateway VTL (virtual tape library) might give you something like what you want. I don't believe Google has a comparable product yet, but I imagine it's coming soon.

The Storage Gateway VTL is an appliance you install on a VMware or HyperV host for $125/mo. You connect it to the Amazon S3 cloud, assign it some local cache storage (for low latency data) and some storage for an upload buffer and then it exposes a media changer and tape drives via iSCSI that you can attach through VEEAM. From the Amazon console, you then create virtual tapes which you can import into VEEAM and use just like normal tapes. The virtual library can hold 1500 tapes, and while the tapes are in the library, the data is stored in S3 at $0.03/GB/mo. If you export the tape from the library, it is moved to the "Virtual Tape Shelf", which transfers the data from S3 to Glacier, where it costs $0.01/GB/mo, the same as Google nearline.

Tapes in the Virtual tape Shelf at Glacier can later be imported back into the library as read-only tapes, but the download costs $0.09 per gigabyte and retrieval is slow. Also, because Veeam cannot perform direct restore from tape, restoring even a single file from a virtual tape would require retrieving all tapes in the backup set, writing all of the data to local disk and then performing the restore from disk. This means that you need a large enough cache to hold all of the tapes at once, plus additional disk storage to write it all back again.

I would consider VTL a good solution for highly redundant offsite storage of data that you plan to never access again, but are required by policy to retain. But its not especially cheap and if you ever do have to retrieve an archived tape from Glacier, it can potentially be very expensive and time consuming.

Amazon Announces Gateway-Virtual Tape Library (Gateway-VTL)
AWS Storage Gateway Pricing
tommls
Enthusiast
Posts: 69
Liked: 11 times
Joined: Apr 28, 2015 7:52 pm
Full Name: Tom Lyczko
Contact:

Re: Google NearLine

Post by tommls »

Did Google (or ANYone) ever come out with anything to make Google cloud storage more 'compatible' with Veeam??

The most I can see so far is to copy a backup copy job's contents to Nearline or DRA, then bring it back down to local storage when/if necessary to restore therefrom.

Thank you, Tom
mongie
Expert
Posts: 152
Liked: 24 times
Joined: May 16, 2011 4:00 am
Full Name: Alex Macaronis
Location: Brisbane, Australia
Contact:

Re: Google NearLine

Post by mongie »

Tom, it can be slightly more automated than that...

I believe TwinStrata (and probably others) support Google Nearline, however there is a difference between Google Nearline + iSCSI Gateway and Veeam Cloud Connect.

With Nearline + iSCSI Gateway, your cloud storage appears as a normal Windows volume, and the gateway manages the translation from Block to Object storage. This is good in theory, but if you have a 1TB Backup file on your cloud storage and need to perform a file level recovery, you must download the whole backup file first (this is handled by the iSCSI Gateway, but is normally slow).

Veeam Cloud Connect is a less convenient, more expensive, channel based solution where the vendor can offer cloud storage (and in V9, replication and VM hosting I believe) along with some Veeam smarts in the cloud. This allows you to mount the backup files on the internet and only retrieve what you need. You can also use the WAN Optimisation feature to reduce the amount of traffic sent to the cloud provider.


Here is the calculation you have to do to find out whats best for you.

Google Nearline is 1c/GB. (Backblaze now offer cloud storage for $0.005c/GB). You also need a cloud gateway. The AWS gateway is $125/month, but it only works with AWS. There are 3rd party options available for other providers - EMC Twinstrata Cloud Gateway, NetApp AltaVault, etc.

Veeam Cloud Connect tends to be ~ 12c/GB+ (and sometimes there are per VM fees too)

If you have a very fast internet connection (actual speed required will depend on the size of your backup storage, but 100Mb+), the Google/AWS/Backblaze might work for you. It can certainly work, but you have to pay attention to the restore time. Veeam Cloud Connect will also work well, but can be 10x more expensive.

If you have a slow internet connection (Less than 100Mb), you could use Veeam Cloud Connect, but keep in mind your full restores are going to take time.



I hope this brain dump can help you.,
Meyercord
Enthusiast
Posts: 36
Liked: 6 times
Joined: Jul 14, 2014 4:31 pm
Full Name: AJ Meyercord
Contact:

Re: Google NearLine

Post by Meyercord »

mongie wrote: Veeam Cloud Connect is a less convenient, more expensive, channel based solution where the vendor can offer cloud storage (and in V9, replication and VM hosting I believe) along with some Veeam smarts in the cloud. This allows you to mount the backup files on the internet and only retrieve what you need. You can also use the WAN Optimisation feature to reduce the amount of traffic sent to the cloud provider.
Do you know if this can be "faked" with something like a cloud-hosted VM running a VEEAM proxy with WAN optimization and high-speed connection to backend cloud storage? Maybe something like an AWS EC2 instance with S3 storage buckets? Would this approximate the Cloud Connect features while allowing vendor-neutral storage, or is there some other benefit to the Cloud Connect service that would be missing from this arrangement?
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Google NearLine

Post by Gostev »

mongie wrote:Veeam Cloud Connect is a less convenient
I would argue this part - as according to all polls, number one reason (and by far) for users to choose Veeam Cloud Connect over alternatives is specifically convenience and simplicity (due to the complete integration).

Overall, comparing Veeam Cloud Connect with "dumb" cloud storage will probably make a good VeeamON breakout session due to the amount of factors and features to be considered. You are listing a few VCC benefits, but there are definitely more to consider. For example, other solutions will not offer you periodic in-cloud backup health check with automated remediation (self healing), which is pretty important if you ask me - because what is the point in backups if you can't recover from them?

Even more importantly, the restore time... you have already mentioned the issue of having to download the entire backup file chain before the restore can be initiated with alternative solutions. With Veeam Cloud Connect, you can restore just what's needed directly from the cloud (single VM, single file and even an application item).

And there are many, many more considerations - both technical and procedural... I am not even attempting to have the detailed discussion on this topic over forum posts ;)

Bottom line, looking at price alone is a wrong thing to do... if you are looking to save money - just don't do cloud backups at all. But if you decide to invest in cloud backups, you'd better make sure you are not throwing your money away for nothing (even if little money).

Nevertheless, it is a very useful brain dump, so thanks for taking time to post this!
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Google NearLine

Post by dellock6 »

Well, the list of things you need to deploy yourself in a VM running in a cloud provider to make it a remote repository, by itself are already a lists of things you do not have to do with VCC. And that VM also needs to be managed once it's running (monitoring, patching, check for free space left and add additional disks to a point where a VM in the cloud usually cannot add more disks). This is comparable to having a server in a colocation, just in the fashion of a virtual machine. But still you have capex to pay for. With VCC, it's a pure pay-as-you-go.

Also, another important aspect that we always list is connectivity: you need to setup VPNs to reach VMs in the cloud, while with VCC you only need a straight TCP connection over the internet.

Finally, price is heavily dependant on the business plan of the provider offering Cloud Connect, but I've seen some providers able to offer really low prices per GB. And as Anton said, pure price per GB is not a fair comparison - even due to additional in-cloud compute requirement for VCC, which increases costs for a service provider.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Google NearLine

Post by veremin »

Also, it's not a fair comparison, especially, when someone operates with the costs given by S3 or Glacier. While doing that, overwhelming majority of users tend to forget about the prices they will be charged with whenever the restoration need arises.

I'm not agitating for one solution or the other, just keep in mind that sometimes there is something else hiding behind low prices, such as download traffic prices or similar.

Thanks.
tommls
Enthusiast
Posts: 69
Liked: 11 times
Joined: Apr 28, 2015 7:52 pm
Full Name: Tom Lyczko
Contact:

Re: Google NearLine

Post by tommls »

I think Nearline may be more useful for storing the results of backup COPY jobs aka 'archive' jobs than backups (fulls with incrementals), e.g. having the archives on local storage plus an offsite copy of the archives.
Then the file sizes and Internet speeds become the issue.
Thank you, Tom
ITP-Stan
Service Provider
Posts: 201
Liked: 55 times
Joined: Feb 18, 2013 10:45 am
Full Name: Stan (IF-IT4U)
Contact:

Re: Google NearLine

Post by ITP-Stan »

Have you tried the Backup Copy Job "Merge" over a WAN connection without accelerator or Veaam Cloud Connect?
The merge takes a lot of time and a lot of bandwidth. I don't think it really is a viable solution except for very small backups.
tommls
Enthusiast
Posts: 69
Liked: 11 times
Joined: Apr 28, 2015 7:52 pm
Full Name: Tom Lyczko
Contact:

[MERGED] Offsite backups to Google Nearline??

Post by tommls »

Has anyone figured this out yet??
There's another thread on this topic with no conclusive answers.
Is it simply cheaper and better to use MS Azure since the cloud connect is included with the enterprise version and one can set it up without a consultant or 3rd-party company...
Thank you, Tom
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Google NearLine

Post by veremin »

As far as I'm concerned this type of storage can be used only in a combination with a 3-party software exposing it as ordinary system, such as NFS, ISCSI, CIFS, etc.

In future, kindly, don't create a separate thread for questions already discussed before, it's more convenient to have feedback regarding one particular functionality within one topic.

Thanks.
primaryeng
Lurker
Posts: 2
Liked: 1 time
Joined: Feb 10, 2018 11:07 pm
Contact:

Re: Google NearLine

Post by primaryeng »

We are running Veeam and creating our backups to local storage and then pushing the VM images out to Google Nearline using Cloudberry (https://www.cloudberrylab.com/backup/wi ... erver.aspx) . Works well and provides the outcome we need. Just wish Veeam would partner up and make it a straight shot out.
ITP-Stan
Service Provider
Posts: 201
Liked: 55 times
Joined: Feb 18, 2013 10:45 am
Full Name: Stan (IF-IT4U)
Contact:

Re: Google NearLine

Post by ITP-Stan »

In my opinion your best option is looking at a (Windows) cloud server with the needed amount of storage attached to it and connected with VPN (perhaps OpenVPN).
The cloud server would then act as a DATA mover for the repository you create, and merges would process on that server locally.
Depending on your licensed version you could also deploy WAN accelerators to speed up the DATA moving over slow WAN links.

Off-course when you choose a Cloud Connect repository everything is done for you without the need for setting up and managing.
Post Reply

Who is online

Users browsing this forum: No registered users and 208 guests