Seeding remote datastore

VMware specific discussions

Seeding remote datastore

Veeam Logoby Tomsyr » Wed Apr 08, 2015 8:40 pm

I'm having issues with setting up an off-site target. I've been through the instructions, but still no luck.
I created a backup copy job that went to a usb external disk. It was sent to the remote datacenter, and uploaded to the target server. I've set up the WAN accelerators, and followed these instructions:
http://www.veeam.com/kb1856

The problem is that WAY too much data is being transferred. I seeded with 2.2 TB's of data, and so far, I've had to 'sync' about 500 GB over the wire, and the job progress is showing that it has a long way to go. I put in a support ticket, but I'm being told that everything seems to be working and everything is fine.
Case # 00873219
In the time between creating the backup and up till yesterday, the amount of changes based on the incrementals equal about 35 gigs.

any suggestions?
Thanks!
Tom
Tomsyr
Influencer
 
Posts: 22
Liked: never
Joined: Tue Jul 01, 2014 3:39 pm
Full Name: Tom Conklin

Re: Seeding remote datastore

Veeam Logoby Gostev » Thu Apr 09, 2015 12:31 am

Hi, Tom

Yes, it does not seem right from your description.

I recommend performing the seeding test locally first:
1. Create a new seed.
2. Put it into some other local storage.
3. Create a new backup job mapping into the seed, and run it (note Transferred size).

This way, you can ensure you are doing everything correctly. Obviously, if your first incremental size will be more than a few GB in a matter of hour since seed creation, then something is wrong.

Thanks!
Gostev
Veeam Software
 
Posts: 21385
Liked: 2348 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Seeding remote datastore

Veeam Logoby Tomsyr » Fri Apr 17, 2015 10:39 am

Hi Gostov,
I was able to do a successful test on a small VM. I sent new copies down to the datacenter, and for a smaller site (500 GB) I was able to get to work correctly. It took a very long time for the fingerprint process to finish.
I started the larger site yesterday, and it looks like the fingerprint process went very slow as well, and at midnight it failed. I suspect that it was because I have my backup are copies set to run at midnight daily. I disabled backups of all jobs (not copies). I figured that since backups didn't run, it wouldn't need to copy.
Anyway, the remote job kicked off again, and I'm back to where I was last week - massive backup going over the wire, and it's not using the seed.
I had to cancel the job.
Is there anyone at Veeam support that can help me with this? We have invested 16 sites into using the Enterprise version of the product. I'm going to upload the logs from the current job.
Also - how can I restart the process without having to create yet another new backup copy to send to the remote data center. Can I put the original metadata file with the full backup file and delete all the current time stamped files?
It seems like if I can get the fingerprint process completed, it will work as needed.
Any estimate on how long 2.2 TB's should take for the fingerprint creation?
Thanks,
Tom
Tomsyr
Influencer
 
Posts: 22
Liked: never
Joined: Tue Jul 01, 2014 3:39 pm
Full Name: Tom Conklin

Re: Seeding remote datastore

Veeam Logoby foggy » Fri Apr 17, 2015 12:07 pm

Tomsyr wrote:Is there anyone at Veeam support that can help me with this?

You can ask your support engineer to take a look at this.

Tomsyr wrote:Also - how can I restart the process without having to create yet another new backup copy to send to the remote data center. Can I put the original metadata file with the full backup file and delete all the current time stamped files?

I believe this is possible.
foggy
Veeam Software
 
Posts: 14716
Liked: 1075 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Seeding remote datastore

Veeam Logoby Tomsyr » Mon Apr 20, 2015 9:35 am

I've spoken to an engineer about seeding remote datastores, and was told that it does take a very long time to create fingerprints. Each disk needs to be created for each VM - In my particular case right now, it's 3 disks. Drive 1 is small, so just a few hours for that. Drive 2 much bigger (1.6 TB) took 24 hours. Disk 3 failed at 79%. Since it was not able to finish the digest, it reverted back to pulling all data across the wire for disk 3. 65 hours into the job, I'm at 88% completed, and now the bandwidth is impacted for production while I wait for this to complete.

We are not using SSD for the global cache on either end, which seems to be the ONLY mention about improving performance. Is there anything else to consider? As mentioned, we invested quite a bit on Veeam for 16 sites - specifically so that we can back up remotely. We've spent a a good deal of time upgrading sites to V8 to take advantage of WAN performance. What else can we do?
Tomsyr
Influencer
 
Posts: 22
Liked: never
Joined: Tue Jul 01, 2014 3:39 pm
Full Name: Tom Conklin

Re: Seeding remote datastore

Veeam Logoby foggy » Mon Apr 20, 2015 12:37 pm

Tom, just to check, what kind of repository do you have on remote side?
foggy
Veeam Software
 
Posts: 14716
Liked: 1075 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Seeding remote datastore

Veeam Logoby Tomsyr » Mon Apr 20, 2015 8:52 pm

Sorry for the late response - I didn't get a notification.
We are running EMC VNX SATA disks.
Tomsyr
Influencer
 
Posts: 22
Liked: never
Joined: Tue Jul 01, 2014 3:39 pm
Full Name: Tom Conklin

Re: Seeding remote datastore

Veeam Logoby foggy » Mon Apr 20, 2015 9:01 pm

I mean, how it is added to Veeam B&R console (what type of backup repository)?
foggy
Veeam Software
 
Posts: 14716
Liked: 1075 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Seeding remote datastore

Veeam Logoby Tomsyr » Tue Apr 21, 2015 9:49 am

Not sure if I am following you...
It was a new backup copy job where the target was a USB drive. After creating, I disabled the job so it wouldn't run again. After the contents of the disk were copied to the remote target, I did a re-scan of the remote repository, then mapped it within the backup copy job I had disabled. I followed these instructions:
http://www.veeam.com/kb1856

I really didn't do anything at the remote site's Veeam console except for rescanning the repository there too. Is there anything that should be done on this console?
Tomsyr
Influencer
 
Posts: 22
Liked: never
Joined: Tue Jul 01, 2014 3:39 pm
Full Name: Tom Conklin

Re: Seeding remote datastore

Veeam Logoby foggy » Tue Apr 21, 2015 9:59 am

Tomsyr wrote:I did a re-scan of the remote repository, then mapped it within the backup copy job I had disabled.

I was juts wondering what kind of repository it is (Windows/Linux server, CIFS)?
foggy
Veeam Software
 
Posts: 14716
Liked: 1075 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Seeding remote datastore

Veeam Logoby Tomsyr » Tue Apr 21, 2015 12:14 pm

Gotcha - Windows...
Tomsyr
Influencer
 
Posts: 22
Liked: never
Joined: Tue Jul 01, 2014 3:39 pm
Full Name: Tom Conklin

Re: Seeding remote datastore

Veeam Logoby foggy » Wed Apr 22, 2015 11:18 am

And Windows server is in the same location with EMC device, right? What about WAN accelerator?

It really depends on the target storage I/O capabilities (hence, SSD recommendations), but I would say that 24 hours that it took to calculate digests for 1.6 TB of data is an acceptable speed (18 MB/s).
foggy
Veeam Software
 
Posts: 14716
Liked: 1075 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Seeding remote datastore

Veeam Logoby Tomsyr » Fri Apr 24, 2015 10:05 am

I'm now trying another site - 35 hours into it, with backups turned off so that is does not interrupt the process. Processed 643 GB out of 2.5 TBs.
Obviously not acceptable.
If I were able to stage the backups on SSD in the remote datacenter, would that fix the performance issues, or is it needed on the source site as well?
I could probably do that for the remote site, but not for the source sites. The console is saying the bottleneck is the Target WAN.
I'm thinking that I could present the drive, get the site to sync, then move it to the slower storage.
The documentation only mentioned SSD for the WAN cache.
Am I missing something? What are others doing to use remote datacenters? Is the initial time to sync just something that has to happen?
Thanks,
Tom
Tomsyr
Influencer
 
Posts: 22
Liked: never
Joined: Tue Jul 01, 2014 3:39 pm
Full Name: Tom Conklin

Re: Seeding remote datastore

Veeam Logoby foggy » Fri Apr 24, 2015 12:50 pm

Tomsyr wrote:If I were able to stage the backups on SSD in the remote datacenter, would that fix the performance issues, or is it needed on the source site as well?
I could probably do that for the remote site, but not for the source sites. The console is saying the bottleneck is the Target WAN.

Yes, in this case getting SSD disks for target WAN accelerator should increase performance.
foggy
Veeam Software
 
Posts: 14716
Liked: 1075 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Seeding remote datastore

Veeam Logoby Tomsyr » Mon Apr 27, 2015 5:17 pm

We're still having issues after a weekend of using the SSD. Is there professional services that are available for Veeam? Anyone to recommend?
Thanks,
Tom
Tomsyr
Influencer
 
Posts: 22
Liked: never
Joined: Tue Jul 01, 2014 3:39 pm
Full Name: Tom Conklin

Next

Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 19 guests