Host-based backup of VMware vSphere VMs.
Post Reply
Dave-Departed
Expert
Posts: 167
Liked: 6 times
Joined: Feb 29, 2012 3:11 pm
Contact:

Target refused global dedupe session?

Post by Dave-Departed »

Hi All,

Just checking through our job logs this morning, as usual, and I see something in one of our backup copy job logs which says:

08/10/2013 06:30:51 :: Target refused global dedupe session because of growth failing. Disable global dedupe.

Never seen this before. Any idea what it means, and why it's happening?

Thanks in advance.

Dave
veremin
Product Manager
Posts: 20413
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Target refused global dedupe session?

Post by veremin »

Hi, Dave. Looks like you don’t have enough space at target location. This prevents target cache from growing, and, hence, there is a corresponding error. The issue should be resolved by allocating additional space. Thanks.
Dave-Departed
Expert
Posts: 167
Liked: 6 times
Joined: Feb 29, 2012 3:11 pm
Contact:

Re: Target refused global dedupe session?

Post by Dave-Departed »

Hi Vladimir,

No more space to allocate, unfortunately! So this leads me to a question...

Our normal onsite backup jobs are set to keep 90 restore points, and this only uses up 5TB in total on our main onsite backup storage. Yet, our offsite storage has 3TB and we have only set our backup copy jobs to keep 7 restore points, and they have almost used up all of the 3TB available! I would've thought that using WAN accelerator and compression, the backup copy job files would be even smaller than the original backup job files which they are created from? Why are our backup copy job files taking up so much space? :(

Thanks

Dave
veremin
Product Manager
Posts: 20413
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Target refused global dedupe session?

Post by veremin »

Hmm, then, I’m wondering what the size of target cache is. Thanks.
Dave-Departed
Expert
Posts: 167
Liked: 6 times
Joined: Feb 29, 2012 3:11 pm
Contact:

Re: Target refused global dedupe session?

Post by Dave-Departed »

We have 2 remote WAN accelerators configured, both with 100GB limits set.

The first is currently at 203GB(?! not really sure why/how?) and the second is at 94GB.
Dave-Departed
Expert
Posts: 167
Liked: 6 times
Joined: Feb 29, 2012 3:11 pm
Contact:

Re: Target refused global dedupe session?

Post by Dave-Departed »

And if I look at the same day for a backup job and a backup copy job, I see this:

Backup job file: 12GB
Backup copy job file: 21GB

Any idea why this would happen?
foggy
Veeam Software
Posts: 21139
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Target refused global dedupe session?

Post by foggy »

Dave-Departed wrote:Our normal onsite backup jobs are set to keep 90 restore points, and this only uses up 5TB in total on our main onsite backup storage. Yet, our offsite storage has 3TB and we have only set our backup copy jobs to keep 7 restore points, and they have almost used up all of the 3TB available! I would've thought that using WAN accelerator and compression, the backup copy job files would be even smaller than the original backup job files which they are created from? Why are our backup copy job files taking up so much space? :(
Dave, I would say it is expected. Backups produced by backup copy jobs are just ordinary backup files, similar to those produced by ordinary backup jobs. WAN acceleration is used for data transfer over the slow link, not for storing data on the repository. Those blocks that are already in cache and are not required to be transferred are read from the cache and written to the corresponding restore point file, so the sizes of backups for both types of jobs should be comparable. Assuming you're using reversed incremental mode in your ordinary backup jobs, full backup is what takes most of the space on both backup repositories and the rest of the space is occupied by incrementals (90 and 7 respectively), which explains the difference in sizes backups take on your repositories (5TB vs 3TB).
veremin
Product Manager
Posts: 20413
Liked: 2302 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Target refused global dedupe session?

Post by veremin »

Dave-Departed wrote:Any idea why this would happen?
What schedule does the backup job have and what synchronization interval is set for backup cop job? Thanks.
Dave-Departed
Expert
Posts: 167
Liked: 6 times
Joined: Feb 29, 2012 3:11 pm
Contact:

Re: Target refused global dedupe session?

Post by Dave-Departed »

Normal backup job runs at 21:00 and takes around 2.5hrs

Backup copy job runs continuously, and sync interval is set at every 1 day, starting at 12:00
Vitaliy S.
VP, Product Management
Posts: 27377
Liked: 2800 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Target refused global dedupe session?

Post by Vitaliy S. »

Might be a silly question - do you have the same set of VMs being backed up and transferred by your backup copy job?
Dave-Departed
Expert
Posts: 167
Liked: 6 times
Joined: Feb 29, 2012 3:11 pm
Contact:

Re: Target refused global dedupe session?

Post by Dave-Departed »

Yes, all of our backup copy jobs are mirrors of our normal backup jobs :)
foggy
Veeam Software
Posts: 21139
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Target refused global dedupe session?

Post by foggy »

Dave-Departed wrote:And if I look at the same day for a backup job and a backup copy job, I see this:

Backup job file: 12GB
Backup copy job file: 21GB

Any idea why this would happen?
Is it also typical for other days?
Dave-Departed
Expert
Posts: 167
Liked: 6 times
Joined: Feb 29, 2012 3:11 pm
Contact:

Re: Target refused global dedupe session?

Post by Dave-Departed »

foggy wrote: Is it also typical for other days?
Yes, give or take.
foggy
Veeam Software
Posts: 21139
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Target refused global dedupe session?

Post by foggy »

Should not be the case, I suggest asking support for assistance.
Gostev
Chief Product Officer
Posts: 31814
Liked: 7302 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Target refused global dedupe session?

Post by Gostev »

Dave-Departed wrote:We have 2 remote WAN accelerators configured, both with 100GB limits set.
The first is currently at 203GB(?! not really sure why/how?) and the second is at 94GB.
Does the first one have more than one WAN accelerator partner? Remember there is a separate cache for each WAN accelerator pair.

As far as the difference in backup sizes, sounds like there are different compression levels on source backup job and backup copy job. Or, target repository has an option to decompress before storing enabled. That is, assuming we are comparing apples to apples: does the source backup job use forward incremental backup mode, and are you comparing the size of incremental backup files (VIB)?
Dave-Departed
Expert
Posts: 167
Liked: 6 times
Joined: Feb 29, 2012 3:11 pm
Contact:

Re: Target refused global dedupe session?

Post by Dave-Departed »

Gostev wrote: Does the first one have more than one WAN accelerator partner? Remember there is a separate cache for each WAN accelerator pair.

As far as the difference in backup sizes, sounds like there are different compression levels on source backup job and backup copy job. Or, target repository has an option to decompress before storing enabled. That is, assuming we are comparing apples to apples: does the source backup job use forward incremental backup mode, and are you comparing the size of incremental backup files (VIB)?
Ahh ok, now it begins to make sense! Yes, the first accelerator has more than one partner (although we eventually separated it out into the 2 accelerators at each end, as I mentioned above somewhere)... So that explains the large cache size!

As for the backup size difference, it looks like we are comparing apples to pears, then... Our regular backup jobs are reverse incrementals (we used to do D2D2T, and we just left them that way when we upgraded to v7) with compression set to high (again, default due to in-place upgrade, as I understand), and our backup copy jobs are forward incrementals, obviously using the 'new' compression and set to 'auto'. I may set the backup copy jobs to 'high', because space is more important to us than using additional CPU resources right now.

Cheers!

Dave
foggy
Veeam Software
Posts: 21139
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Target refused global dedupe session?

Post by foggy » 1 person likes this post

High provides only 10% of additional compression ratio over Optimal level, so do not expect the sizes to be equal after applying new level. 2x difference is mostly explained by different backup modes, I suppose.
Dave-Departed
Expert
Posts: 167
Liked: 6 times
Joined: Feb 29, 2012 3:11 pm
Contact:

Re: Target refused global dedupe session?

Post by Dave-Departed »

I came in this morning to see this message, and wondered if it's related to the above?

Code: Select all

11/10/2013 07:28:52 :: Processing '<VMname>' Error: An existing connection was forcibly closed by the remote host
--tr:Cannot read data from the socket. Requested data size: [1048552].
--tr:Receive file failed
--tr:Wan connection failed to run command.
--tr:Wan connection failed to get target file '{d03e1a8e-cf4b-48a8-83d3-3e508f8dbf3a}'
--tr:Sync global cache failed.
--tr:Disk processor failed to init global dedup.
--tr:Disk processor failed to process disk.
--tr:Sync task failed to process disk.Arguments:
Vitaliy S.
VP, Product Management
Posts: 27377
Liked: 2800 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Target refused global dedupe session?

Post by Vitaliy S. » 1 person likes this post

Looks like it is just an intermittent connection drop, which is not related to the original issue.
Post Reply

Who is online

Users browsing this forum: Baidu [Spider] and 24 guests