Backing up the replication target?

Posted by James Palmer on 01-Feb-2017 05:08

I'm going through an exercise of getting a better understanding of what our clients are doing with their databases. It's led me to a number of very surprising practises, some of which I understand the impact, some of which I don't. This is one of the latter. 

This client have OE Replication running. Because of the performance impact they perceive of backing up the primary database someone has switched them to backing up the replication target instead. 
What are the thoughts on this practise? Is it a good idea or not? Please justify your answers with evidence so I can educate the ignorant (including myself!). 
I have some ideas as to why this is a bad idea, but I want to hear from those who know rather than think it's bad/good. :) 

All Replies

Posted by Tom Cattigan on 01-Feb-2017 07:10

Hi James,

My initial concern would be making sure that production is completely in sync with the target before backing up. A scenario would be if automated, what if the source gets out of sync (say all ai extents filled up and replication stalls) - you blissfully back up the target but then lose source - you would lose data in this scenario.

I am always concerned about touching the target but probkup doesn't seem to break it so you should be good there.

What is the performance issue?  When backing up are you using private buffers?  -Bp 64?  This will make sure the buffer pool is not overloaded although the file system cache will be affected it should not be too bad depending on the usage at the time of course.

Regards,
Tom

Posted by ChUIMonster on 01-Feb-2017 07:33

My main concern is similar to Tom's -- any number of things might result in the target being far behind the source.  So if you are not monitoring that and making sure that they stay connected and reasonably in sync then you could be putting production data at risk.

It would also be nice if you could roll forward after image files from the source against a backup made from the target but the last time I looked into it that isn't actually possible :(  Without that kind of capability you lose a layer of "belt and braces" redundant recovery capability.  And I like having as many different ways to recover as possible.

I too would be curious to know what the specifics of the perceived performance impact on the source are.  Aside from the impact that it will certainly have on the local disk another possibility is that there is, perhaps, a very large bi file.  Depending on what release you are on that could be painful to deal with.

Posted by James Palmer on 02-Feb-2017 05:19

Thanks chaps. I've been in touch with the customer. They're pretty savvy in a lot of ways which is good. They've had a Progress consultant in who has advised on the strategy. Their backup script apparently checks a couple of high use tables for the latest record in source and target and only backs up once the two match. A bit risky, but not as bad as blindly going in.

I'm not sure what the performance issues were, he didn't say, but I highly suspect it's the issue in Windows where the backup takes longer if the target file already exists. Oh, that and the -Bp issue.

I've explained the approach and the improvements seen at other sites and await his reply. He agrees with Tom B that trying to work out how to apply AI to a replica backup in the heat of the moment is probably not a good idea and one they should at least attempt to do. He has a KB article on the issue which certainly helps the cause for avoiding it! knowledgebase.progress.com/.../000038993

Posted by George Potemkin on 02-Feb-2017 05:49

> Their backup script apparently checks a couple of high use tables for the latest record in source and target

Would it be better to check the last TRID?

It's sad that probkup and prorest do not report the last TRID for db's snapshot though it's stored in the backup's header (and, of course, in the master block).

Posted by James Palmer on 02-Feb-2017 05:56

I'm sure that would be better. It's not my script though and I don't have the authority to change it, unfortunately. I can only advise.

Ideally I want to get them back to backing up the primary. So I'm trying to understand what the issues were.

Posted by James Palmer on 03-Feb-2017 10:58

Interesting how a story unfolds. It turns out the reason for the switch of backup strategy is down to the fact that there's not much disk space left on the source server. Further investigation shows they have 250GB of AI files. Their AI was set up before we had anyone familiar with the AI Archiver, and is a batch job that switches the AI and then copies the full ones off somewhere. I'm guessing it works ok, except they have fixed AI extents and so 2.5 days of AI files is rather a lot of wasted space.

Switching them to use the AI Archiver, and switching the backups back to source should be a trivial job once they are convinced!

Posted by Tom Cattigan on 03-Feb-2017 11:18

Excellent - nice to see an easy resolution :-)

Posted by cjbrandt on 03-Feb-2017 11:55

if someone is using fixed ai extents and not using the AI Archiver...  There is an option to extract the data from the AI extent rather than copy the entire extent.  So if you have a 500mb AI extent with 12mb of data in it, you only archive 12mb of data vs 500mb.  

This thread is closed