Backup Copy Job and transforming full backups

[soylent]

I have a Backup Copy Job set up to copy daily backups to a deduplication appliance. Since it's inline dedupe, it's extremely slow for reading, but it's fine for just writing, so I only use it for backup copy jobs.

Initially I had the job set up with just 6 restore points, no archive. Since it was just copying restore points, the performance was fine. Now I am trying to set it up so that I keep weekly and monthly restore points. But when I enabled that, the job is now transforming full backups on the repository itself, which takes like 20 hours.

I just want to prevent it from deleting weekly and monthly fulls, not create them. How can I set this up so that it doesn't need to transform full backups?

Here is how I have the job set up:
Copy every 2 days at 1AM.
Processes 4 backup jobs (each run daily and are linked)
Restore points to keep: 6
Keep the following restore points: 2 weekly (Sunday 12:01AM), 3 monthly (first Sunday of the month)

Thanks!

[foggy]

Transformation is an essential part of the backup copy process, since backup copy job doesn’t copy backup files per se but rather synthetically creates required restore points in the target location from VM data stored in source backup repositories. On reaching the specified number of restore points to keep, backup copy job performs transformation during each run. To avoid transformation required to keep GFS restore points, you could use a simple script that, being scheduled via Windows Scheduler, will copy the latest full backup to the required location on weekly/monthly basis.

[soylent]

Thanks for the reply.

Does the archive feature run a full backup job itself, or does it basically put aside an existing previous backup? And does it archive full restore points only, or does it archive incrementals?

[veremin]

If you're talking about GFS retention settings in backup copy job, then, VB&R uses a full backup created in the regular backup cycle to create required GFS restore point (weekly, monthly, yearly, etc.) Once the specified retention policy is exceeded, the full backup starts moving forward on regular basis. When the full backup reaches the day chosen as a day for GFS retention, the new GFS point is created from it.

See our online Help Center for more information regarding GFS retention scheme.

Thanks.

[lightsout]

I use a deduplicator too, and I find this process very difficult to manage as a result. I run an active full weekly, but the backup copy won't copy it for the GFS and keeps doing an extremely long transformation each day instead.

So I'd be much better from my POV is the backup copy would copy the full backup, if it existed, rather than doing a transformation each day. I can't just replace my expensive deduplicator easily to achieve the I/O required for the transformation.

Another option would be if the VM copy job had a GFS schedule built into it as well.

[veremin]

As mentioned above, the transformation is essential part of backup copy job activity. Also, the backup copy job has its retention setting (number of restore points to keep) that don't rely on settings of the source backup job, thus, the transform operation can't be replaced with copying "source" full backup. Let alone the fact that backup copy job isn't about direct restore point copy, but it's more about synthetic restore point creation (explained by Alexander). Thanks.

[lightsout]

And that is ok, but what do you suggest as a workaround? If the transformation takes too long (days!) for me to use a backup copy job, then I have a problem.

I'd be happy with a VM job copy with a GFS schedule for example.

Otherwise I'm going to have to script copying of my vbk files, which will work, but not really desirable as I'm sure you'll understand.

[Vitaliy S.]

I'd be happy with a VM job copy with a GFS schedule for example.

But in this case all your disks will be converted to thick ones (as thin disks can only be stored on VMFS volume) and you will consume a lot more space compared to deduped and compressed backup file.

How many backup copy jobs are running at the same time? If your target device does not have enough IOPs, then you can use regular backup jobs to save your backups for archival purposes.

[lightsout]

Even running just one job takes an awfully long time. 1Gbps max network speed doesn't help.

You're right, in that I could configure a backup job to run say monthly. The issue is see is when the monthly runs, it is difficult to stop the weekly running at the same point, and I'd prefer to avoid having two full backups ran.

[Vitaliy S.]

Ok, thanks for the feedback, it seems like you can go back to "old-school" method of copying backup files to the dedupe storage. BTW, what is the make and model of your target storage?

[veremin]

The issue is see is when the monthly runs, it is difficult to stop the weekly running at the same point, and I'd prefer to avoid having two full backups ran.

You can write a script that will check whether or not the weekly job is running and execute or postpone the monthly one in accordance. Thanks.

[lightsout]

My storage is a Quantum DXi 6520. So an older model, but I'm limited by its 1Gb ports so it can certainly ingest data at 100MB/sec all day long. Reading data to tapes is also fine, but the random I/O of the transformations is just awful, so much so that my transformations can just hang.

Regarding the other comments, I'm just going to copy the files by script to the dedupe device in the meantime.

I guess what I'd like in the future is a job that runs like the tape job, that just copies existing backups, but can do the GFS schedule as well.

[b.vanhaastrecht]

Hi,

We've had a couple Veeam setups were we had trouble with copy jobs and GFS to a dedupe unit. The problem lies in the inflexibility of the copy job GFS method. It currently has one method of creating a new full, and thats thru a synthetic process. This IO patern does not work well on dedupe units. Offcourse, if you can comply with your RPO depends on how much data you got and the processing speed of your dedupe unit. In Veeam v8 aditions are coming to speed up the copy and transform by using source side deduplication, but only with specific vendors.

With this adition, it will become faster. But still, transforming is synthetic. And until this point, this adition is not coming for Quantom DXi.

Having an option to do create a GFS by "full" instead of "synthetic" would solve this in some cases. Just copy the data again would be much faster then synthetic creating. Thats actualy what copy means right

(Please dont suggest scripting, I know this is posible, but scripts does not give you full control as via the GUI).

Thank you, regards,
Bastiaan

[Vitaliy S.]

Hi Bastiaan,

Yes, this feature has been requested before (in the posts above), so thank your feedback as well.

[lightsout]

My solution has to be create separate jobs to go to the Quantum. This is painful as it hits the production VMs twice, and obviously can even block the other backups. So I really hope v8 will help our backup copy jobs work better for us.