WAN accel not adhering to cache size limit

VMware specific discussions

WAN accel not adhering to cache size limit

Veeam Logoby emk23 » Fri Jun 13, 2014 2:02 am

Noticing our target WAN accelerator is creating an ever expanding .tmp file in VeeamWAN\Send\Genfiles\guid\

Why is it doing that?

What is the point of having a configurable WAN accel cache size maximum if the server is going to create these other files? It chewed up all the space that we defined for the parameter, now it is using up all the rest of the space.

Frustrated because we have been pounding our heads against the table trying to get a 2TB remote site backed up to our DR site with copy-job+WAN Accelerator.

We have literally been spending weeks sending USB disks back and forth, and when we finally think we are "there" we encounter a out of control temp file consuming all of the disk space on our target WAN accelerator. Job has been calculating digests for 10+ hours and before it finishes that I am sure the disk will run out, and we will be back to square one - tearing our hair out trying to get our remote sites backed up. :evil:
emk23
Influencer
 
Posts: 11
Liked: never
Joined: Mon Jul 23, 2012 1:30 am
Full Name: Eric

Re: WAN accel not adhering to cache size limit

Veeam Logoby emk23 » Fri Jun 13, 2014 5:40 am

Well, the job chewed up 40GB of disk space for no reason with these temp files and then failed at 99% with a "failed to create disk digests" error message.

Venting right now, but I am generally PO'ed with how difficult it is to get large source site VM's to my DR site using Veeam. Everything else about the product works well except copy-job+wan-accelerator.
emk23
Influencer
 
Posts: 11
Liked: never
Joined: Mon Jul 23, 2012 1:30 am
Full Name: Eric

Re: WAN accel not adhering to cache size limit

Veeam Logoby v.Eremin » Fri Jun 13, 2014 10:57 am

What is the size of the source VM data? Have you taken into account 20 GB overhead required for each 1TB of source VM data (Wan Accelerator System Requirements)? Thanks.
v.Eremin
Veeam Software
 
Posts: 13296
Liked: 974 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: WAN accel not adhering to cache size limit

Veeam Logoby emk23 » Fri Jun 13, 2014 12:29 pm

The source site (single job) contains approximately 3TB of VM data.

Apparently we did not see the "Disk Space: Global cache size as defined by user, plus 20GB per each 1TB of source VM data."

It would be nice if this was more at the forefront of the configuration screen. :|


The real question is - how do I force the job to restart the creation of the digest data?

The target wan accel host became unstable due to lack of disk space and I had to stop the copy job.

Now that I have resolved the disk space problem and restarted the job - it seems that the job is not recreating that digest data or using the data that we seeded to the target site.
emk23
Influencer
 
Posts: 11
Liked: never
Joined: Mon Jul 23, 2012 1:30 am
Full Name: Eric

Re: WAN accel not adhering to cache size limit

Veeam Logoby emk23 » Fri Jun 13, 2014 3:33 pm

I was able to trick the job to recalculate digests by setting up a temporary repository on a tertiary machine we were using to copy the seed file from USB disk to the target repository - then pull out the big vbk to a different folder, purge the folder and rescan repository on the target repository, the move the vbk back in and bring only the vbm off of the seed disk.

I then pointed the copy job back at the target repository and it is again now recalculating digests.

It would be nice if Veeam could figure a way to make it "resume" digest calculation in the event of a problem. I reviewed the logs and it stated that the changed extents were not available and it was doing a full.


Last question - if I change a copy job settings from one target WAN accelerators to a different one, both in the target data center (and the target repository does not change) is a digest recalculation done? Is the digest information stored on the WAN accelerator or the target repository?
emk23
Influencer
 
Posts: 11
Liked: never
Joined: Mon Jul 23, 2012 1:30 am
Full Name: Eric

Re: WAN accel not adhering to cache size limit

Veeam Logoby foggy » Mon Jun 16, 2014 1:08 pm

emk23 wrote:Last question - if I change a copy job settings from one target WAN accelerators to a different one, both in the target data center (and the target repository does not change) is a digest recalculation done? Is the digest information stored on the WAN accelerator or the target repository?

Digests are stored on the source WAN accelerator, so changing the target one will not cause their re-calculation (however target WAN cache will have to be populated using data stored in the target repository).
foggy
Veeam Software
 
Posts: 14765
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Sizing WAN Accelerator Cache

Veeam Logoby krismason » Mon Sep 01, 2014 4:51 am

Hi There,

Is there somewhere that provides any more information on the sizing of WAN Accelerator cache? 10GB per OS a mentioned on the window and the information about multi (branch/parent office) locations isnt really that helpful?

Thanks

Kris
krismason
Influencer
 
Posts: 13
Liked: never
Joined: Sun Feb 16, 2014 9:16 am
Full Name: Kris Mason

Re: Sizing WAN Accelerator Cache

Veeam Logoby Shestakov » Mon Sep 01, 2014 9:56 am

Hi, Kris,

You can find more info in the User Guide`s section about WAN Cache.
Here on the forum we have some relevant topics: one, two.

Also, please review system requirements and mention that additional 20GB of space is required for each 1TB of source VM data.

Do you have any specific question?

Thank you.
Shestakov
Veeam Software
 
Posts: 4865
Liked: 396 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: WAN accel not adhering to cache size limit

Veeam Logoby EdwardsCP » Tue Oct 21, 2014 7:52 pm

Bumping an old thread, but this is relevant to an issue I'm having with WAN Cache sizing. Can I have some clarity on the best way to determine how much disk space the WAN Cache requires? At one point I was told cache should be 100 GB plus 10 GB per each different OS that's backed up. Now I've found the "plus 20 GB per each TB of VM data".

Let's say, hypothetically, you have 6 different OS's being backed up and a total of about 2.5 TB.
100 GB base-requirement for cache
plus 60 GB for OS's
plus 50 GB overhead (20 GB per TB of data)
210 GB total?

Is that an accurate way to size your WAN Accelerator disks?


Edit: this should probably be right in the B&R forum...not necessarily the vSphere subforum, but I just found the thread from a search.
EdwardsCP
Influencer
 
Posts: 15
Liked: 1 time
Joined: Tue May 20, 2014 8:12 pm
Full Name: Colin Edwards

Re: WAN accel not adhering to cache size limit

Veeam Logoby v.Eremin » Wed Oct 22, 2014 8:57 am

EdwardsCP wrote:Is that an accurate way to size your WAN Accelerator disks?

The correct approach is mentioned in the system requirements section of our User Guide. The OS reference has most likely to do with initial sizing (when you're planing to define global cache size in GUI), rather than with the overhead.

Thanks.
v.Eremin
Veeam Software
 
Posts: 13296
Liked: 974 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: WAN accel not adhering to cache size limit

Veeam Logoby EdwardsCP » Wed Oct 22, 2014 1:55 pm

Sorry, but I doubt I'm the only one that finds those requirements to be vague. That's why I was asking for clarification.

We followed a recommendation from support (case# 645925) and sized our WAN Cache disks for 100 GB plus 10 GB per OS. A couple of weeks later, we have run into the situation again where the Backup Copy Job can't run because there is no disk space left.

Walking through the hypothetical scenario that I gave and correcting it if I have erred in my understanding of the recommendations would have been more helpful than pointing me at a document that I already read.
EdwardsCP
Influencer
 
Posts: 15
Liked: 1 time
Joined: Tue May 20, 2014 8:12 pm
Full Name: Colin Edwards

Re: WAN accel not adhering to cache size limit

Veeam Logoby foggy » Thu Oct 23, 2014 12:02 pm

Do you have those 20 GB of free space per 1 TB of source VM data on top of the specified cache size? The recommendation is to have, for example, 100 GB cache plus additional 20 GB per 1 TB of data overhead in the cache folder (not included in the specified cache size).
foggy
Veeam Software
 
Posts: 14765
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

[MERGED] WAN Cache 100GB not enough?

Veeam Logoby randy » Thu Nov 06, 2014 4:19 pm

We have 2 Backup Copy jobs that go from site A to B using WAN accelerators. Both jobs are for 3-4 VMs running a mix of 64-bit Win2008r2 and Win2012r2. We have 100GB WAN cache on each accelerator. But we are getting "Target refused global dedupe session because of growth failing. Disable global dedupe." I can allocate more cache but from the documentation it seems like 100 GB should be more than enough at 10GB per OS. Should I just increase cache to 200GB or is there another problem/solution?
randy
Influencer
 
Posts: 12
Liked: 1 time
Joined: Wed Jun 08, 2011 2:05 pm
Full Name: Randy Smith

Re: WAN accel not adhering to cache size limit

Veeam Logoby foggy » Thu Nov 06, 2014 4:29 pm

Randy, please review above for the WAN cache size recommendations.
foggy
Veeam Software
 
Posts: 14765
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

[MERGED] WAN Accelerator. Sizing.

Veeam Logoby nunciate » Thu Feb 05, 2015 9:56 pm

I know there is info out there on this but it is a bit confusing.
I have a backup storage server with about 63Tb of backup data on it (VBK files total doesn't include incremental backups also on the storage).
There are about 230 VMs and I want to backup copy all of them to my DR site.

I currently have the server to be placed in DR connected here next to my primary server.
There is a single Gigbit interface connected to the DR server as that is what we will have as our WAN connection from production to DR.
I have also set throttling on the servers to not exceed 1Gbps

Now I already have all of my backup copies done using direct mode and they have been syncing up nicely over the last month or so.
Before I move the server to DR I would like to turn on WAN Acceleration.

One of my file servers has multiple 2TB drives totaling about 10Tb of storage. Maybe 8Tb used.
I turned on WAN Acceleration for the backup copy job on this server and it filled my cache and failed after a very long time. I didn't write down the times.

Both servers have a 450Gb high performance SSD drive to use for WAN Cache.
I have each WAN Accelerator set to 400Gb but apparently that isn't' enough.

Can someone simplify this for me or tell me how much cache I really need. Honestly I don't know that I even need to use acceleration since I have a gigabit connection to my DR site but every little bit helps as we will have other things going over that pipe as well.
nunciate
Expert
 
Posts: 137
Liked: 22 times
Joined: Tue May 21, 2013 9:08 pm
Full Name: Alan Wells

Next

Return to VMware vSphere



Who is online

Users browsing this forum: jslic and 23 guests