What's the best architecture for desire to have 3 databa

Posted by Admin on 20-Jul-2006 00:59

Hello,

I was wondering if anyone has information on how to sync 3 databases (same schema etc) across continents.

Information entered in 1 database should be updated to the 2 other databases. This needs to be the case for all the 3 databases. (That is some information is also entered at the 2nd installation and this needs to be updated to the 2 other locations. Similarly the data entry and transactions created at the 3rd location to be fed back to the 2 other databases). Basically all the 3 databases have the same structure and schema. But all are at different locations (in fact different continents). Product information entered at central database needs to be updated to the other 2 and transactional information entered at the 2 other locations needs to be fed back.

What is the best architecture for this with Progress ?

Another consideration was to have 1 central database. But then is the connection over VPN (via appservers) reliable over continents ? Anyone out there have similar situation/experience in this ?

Any suggestions?

All Replies

Posted by Alon Blich on 20-Jul-2006 03:04

AppServer. The idea of a distributed database, although its a very interesting one, isn't very practical.

I think, doing something like that is mostly possible with a very small and controlled envronment or choosing a replication scheme without data collisions.

Maybe the best solution for you is a web application and WebSpeed, even if you're not a web shop.

Especially if you're new to distributed application architectures, the OpenEdge reference architecture etc.

Posted by Brian Bowman on 20-Jul-2006 11:57

OpenEdge Replication will do this from a central database to 2 targets, but is not bi-directional (what you are really looking for).

Soon Progress will be offering Bi-directional replication via DataXtend (http://www.progress.com/realtime/index.ssp) . This will perform bi-directional sometimes connected replication.

If you would like more information regarding this please contact me directly. I will forward your information to the appropriate resources.

Brian Bowman

bowman@progress.com

Posted by Admin on 20-Jul-2006 18:59

Thanks for the prompty reply.

1) Appservers consideration: The databases are at remote locations (various continents). So question of VPN access to Progress database comes into play. Dont think WAN is an option for this. Also Progress dont recomment connection over VPN (since it is not reliable).

2) Regarding DataXtend - I dont know much about this? Is it like SONIC ESB ? I assume SONIC ESB is a solution but the price is quite daunting for 3 seperate machines ( about 30,000 Australian dollars: 1 SONIC ESB Enterprise costing about 10,000 Australian dollars).

I am not sure if SONIC Remote suite would be enough for this ? ( SONIC REMOTE SUITE costs about 3000 Australian dollars per CPU ).

With DataXtend - bidirectional replication : Would this require lot of development or is it aware of the transaction data being added and "auto-replicates" them back to the other "dependent" databases ? Or would it require some triggers to be put in database and "data files" to be created to operating system ? Also how would the replication work across continents ? Connect via VPN to an appserver to the other databases ? ( IS VPN connection ok with Progress ?)

Any info on this would be useful to me. Thanks in advance. Or from your experiences, is there a better architecture for this ? I understand that there are some Microsoft "sonic" type products where the cost is very low (or almost free). Anyone aware of these products? Also how have other sites implemented this kind of architecture ( where data is shared amongst 3 similar databases in a distributed enviroment ? )

Regards

Posted by Tim Kuehn on 21-Jul-2006 07:54

Anyone aware of these products? Also how have other sites implemented this kind of architecture ( where data is shared amongst 3 similar databases in a distributed enviroment ? )

I've written and implemented a structure like this - see my response to your message in the User Support forum.

Tim

Posted by Alon Blich on 21-Jul-2006 09:33

Tim, I'm not trying to intrude.

But we are talking about ASYNCHRONOUS log-based, peer-to-peer ("update anywhere") replication model ?

Where any user can update any data in any site and replication happens in set intervals. That has been implemented on a production database with a reasonable size of hundreds of tables ?

I would recommend seeing a customer that has implemented a system before going in that direction.

For that reason and much more, I think, a web or distributed application would be a far better not to mention simpler choice.

Theres an endless examples of secure, large scale web and distributed applications in Progress and anywhere else.

Just my 2 cents.

Posted by Tim Kuehn on 21-Jul-2006 09:49

But we are talking about ASYNCHRONOUS log-based, peer-to-peer ("update anywhere") replication model ?

Exactly.

Where any user can update any data in any site and replication happens in set intervals.

Well, in the code I wrote, the replication was done when the transaction finished. The end-to-end time from when the source transaction was committed to when it was applied to the remote sites was generally under a minute if things were quiet. If there was a lot of activity, it could take longer, but not much.

To implement "any user can update any data in any site" is a potential data collision nightmare, and I think would only be reasonable when using synchronous replication. The chances of introducing logical inconsistencies in the data is just too great otherwise. (In the original poster's case, synchronous replication isn't an option though).

I would recommend seeing a customer that has implemented a system before going in that direction.

I've got a customer who'se implemented the model I created, and are using it in their customer sites with good success.

Tim Kuehn

Posted by Alon Blich on 21-Jul-2006 10:07

I don't understand you're saying it would be a "potential nightmare" but theres no other choice ?

If we're talking about synchronous (over diff continents!) or distribution or consolidation replication models then that would be something else.

But they do come with substantial tradeoffs/limitations.

Whats wrong with a web or distributed application ?

Posted by Tim Kuehn on 21-Jul-2006 10:13

Whats wrong with a web or distributed application ?

The question, as I understand it, is they want transactions done on one location to be replicated to the other sites.

If the application is not designed as a distributed or web app, you could be looking at a potentially huge commitment to re-do the application.

But maybe I've got the wrong idea about what you mean by a "web" and "distributed" application. Perhaps if you elaborated a bit on your exact meaning, I won't end up answering questions you're not asking.

Tim Kuehn

Posted by Thomas Mercer-Hursh on 16-Aug-2006 15:41

From what I saw at Exchange, I would definitely follow up on the possibiltiy of DataXtend. Goodness knows where the cost will end up, but when they have this available for Progress databases, it should provide a very cool capability. In particular, the RE edition is designed for environmens where the connection between the databases may not be reliable, either it is usually connected, like it sounds like you have, or usually not connected, e.g., a remote salesperson.

This thread is closed