OERA,, SOA, OO, Entity, and View

One of the Seattle BizzTalk-meetings (SOA) I attended back in 2000 or so was one of Pat Helland about "the fortress model". It's about not trusting incoming data at every level:

"...

The Helland Model (from Pat Helland, of Microsoft) describes separation based on trust boundaries. He sees an enterprise architecture as based on a collection of "fiefdoms". A fiefdom is a trust boundary and a functionality boundary with its own private data storage. Pat also thinks of the presentation tier as an "emissary" representing browser based clients to the fiefdom.

..."

I borrowed the summary from http://www.objectwatch.com/newsletters/issue_36.htm

from

Another thing he writes about is "Metropolis": "...Pat Helland relates the evolution of technology architectures to the evolution of city planning and shows us how far we have yet to go...". See http://www.pathelland.com/ and http://msdn2.microsoft.com/en-us/library/aa480026.aspx.

In Razorbills, there is presented the notion of

Entities vs. Views. There is a strong relationship

here between Entity and what one would usually think

of as a class in the service which owns that data.

The point is made that the "view" of this data

needed by various consuming services doesn't need to

be the entirety of the data in the entity, nor is it

limited to the data which is within the entity,

i.e., it may require data resulting from a join with

other entities. This creates the interesting notion

that, for example, the AR service, which owns

Customer, might have a Customer entity or class, but

that when the Order service needs Customer data, the

view that it requires is something other than this

complete entity or class. Thus, the AR Service

might have a message destined for the Order Service

which allows the Order Service to build an

OrderCustomer class corresponding to this view. I

think this simplifies some of the issues of class

design which have been raised earlier.

Yep - I've used this sort of thing before (Modelling tools like to use this I've found). Where you can run into trouble if your not careful is creating excessive reads. Lets say we do create an OrderCustomer object. Now lets say we query all order's for day and return the OrderCustomer objects. Now, lets also assume we had 1000 orders and one customer did 990 of them. So, if you don't architect carefully, you end up with a solution where you go:

for each order where :....

no-lock:

find first customer of order no-lock ...

create OrderCustomer object.

assign fields / call methods.

end.

Of course what we end up with reading the same customer 990 times. I've seen this quite a bit in Java OO systems where you make the solution so generic that it loses some of its smarts.

So - as we go down this OO path, I think we are going to need better ways of "caching" objects in memory and linking them directly back to the DB. Maybe we could flag an Object as "like table" and have the DB keep it automatically upto date just like a normal row ??? E.g. have the DB handle all the writing / locking. Then we could simply look up a record (e.g. find customer of object) and put a pointer to the "real" in memory buffer into the ABL runtime. This could them be copied into a complete memory image when its passed outside the "shared memory" ABL pool (e.g. via an appserver).

Anyway - back of track - getting this "business" level objects is great for the business people. Of course there are downsides when you start creating and shipping around lots of objects as the speed of that is a lot less than a straight query from the ABL for SQL.

Others questions to ask are

• how many different types of objects to I want? Do I want a customer, order and ordercustomer, ordercustomerlines, etc or will I leave them on their own?

• When do I return ordercustomer and just order?

• When I create a new order, do I want the end user to send OrderCustomer or just order? If its just "order" how do I find the customer or does "order" contain the unique customer key? If so, how do they get it?

• What happens if my OrderCustomer is out of date? How do I maintain state?

• How do I refactor a change through all the objects / WebServices and ensure they all still line up?

• Can I "version" my objects and WebServices?

A lot of these are covered in the article but I never really ""understood"" them until I tried it for myself ...

Anyway - its all fun and games and its especially good to keep stretching ourselves. Thats why I love these forums

Murray

This is

where I think the second article has some interesting notions. If

you think about it, even in a monolithic application, if the

customer information is read no-lock for inclusion in the order,

there is a certain danger of it being out of date by the time the

order is completed. By and large, that doesn't actually matter

except for information which the Order needs to update, such as

CustomerTotalValueOnOrder and that is going to occur in a

transaction post which occurs as a part of the completion process

and will take the form of "add $1000 to TotalValueOnOrder" not

"Make TotalValueOnOrder = $5000". So, the only kind of updates to

Customer that one needs to worry about are things like changes to

tax authority or an immediate change in shipping address ... but

those are things where one just cant worry about the few seconds of

time in which the order is being processed because the chances of

those update occurring in that interval are vanishingly small. Even

in a monolithic system, it is much more likely that the update to

Customer will happen either before or after the processing of the

Order. As well as being deadly serious.

While I hear your wish, I think it is misplaced. If

the database is local, we already get this service

performed for us by the database buffer. The real

issue is what happens when the authoritative database

is not local.

This has nothing to do with the database or the -B parameter. Its about managing objects, mapping objects to database tables and defining a unit of work. Once you go the route of using classes to represent your entities and when every entity represents a persistent object, than you have to track the objects during the request, the same way as the ABL tracks buffers (subsequent finds are tested against a local buffer context). I'm talking about a Customer class instance being mapped to a customer database row. Why wouldn't the ABL help us with this?

But one step back, which we did earlier, is the question: should we instantiate an ABL entity (class instance) per domain object or should we pass (wrapped) temp-tables... So should we do it the Java/.Net way or the ABL way

By and large, that doesn't

actually matter except for information which the

Order needs to update, such as

CustomerTotalValueOnOrder and that is going to occur

in a transaction post which occurs as a part of the

completion process and will take the form of "add

$1000 to TotalValueOnOrder" not "Make

TotalValueOnOrder = $5000".

What kind of value would that CustomerTotalValueOnOrder be and in which currency?

For me, temp-table

certainly have their place, but if one is writing OOABL, then a

domain object is a class instance. One could elect to have a one

row temp-table inside that class instance, but I am not currently

thinking of that as best practice.

A monetary value in whatever base currency

the application is using as a reference. Are you asking why one

would want such a number?

Well, for starters, because the mapping is quite

likely not to be one object to one row.

No, and I didn't claim so either

Not to

mention, of course, that service A might ask service

B for a Customer view, which is sent via a Sonic XML

message to a remote machine where service A

instantiates a Customer object.

You forgot to mention that the message was transmitted to a satellite, forwarded to ISS and echoed back to earth. This is all middleware. Object persistence is an internal issue. Messages are transmitted in external format and translated back to internal requests.

You expect this to

be linked automatically back to a database managed by

another service on another machine?

Huh? I expect the other service to receive an external request, accept the request and transform the request to internal calls or deny it. When accepted it will delegate the actual work:

- massaging the message

- instantiating domain objects

- invoking operations on them

- assembling the return message

But why make things so complex? Your referenced link makes sense by stating that tightly coupled services should probably have been designed as one service.

For me, temp-table certainly have their place, but if

one is writing OOABL, then a domain object is a class

instance. One could elect to have a one row

temp-table inside that class instance, but I am not

currently thinking of that as best practice.

He, he, that's an understatement. It's a bad design! It's like creating a table for every row you want to store in the database...

What kind of value would that

CustomerTotalValueOnOrder be and in which currency?

A monetary value in whatever base currency the

application is using as a reference.

Are you asking why one would want such a number?

Quite frankly: yes, since it's like adding apples and peers and present that figure as "good". Why would you want to sum the order values and store that figure in the customer? And when you want to shoot that "add $1000 to TotalValueOnOrder" to the other side of the world via Sonic, how will you maintain integrity?

That is

somewhat my reaction as well, but the counter argument is that

temp-tables provide some very useful services like the to/from XML

methods and the before-table functionality for determining what has

changed. In fact, I had myself thinking recently about how many of

the languages I have worked in have had some interesting way of

dealing with problems that is not found in a lot of other

languages, but which one sometimes wants to imitate. And, in

specific, how COBOL was interesting for its ability to define the

same data area in multiple ways. Suppose, for example, that one

could define properties for a class, but that one could also define

a buffer which contained those properties and the buffer had XML

methods and before-table functionality. Wouldn't that be neat? We

must have

And what

is your basis for deciding that a customer has sufficient credit

available to ship an order. To me, this is typically something like

Credit Available = Credit Limit - (Total Open Invoices + Total

Orders Ready to Ship) If the current order which is being proposed

for authorization has a value less than or equal to the Credit

Available, it is OK to authorize it. Then, its value needs to be

added to Total Orders Ready to Ship before evaluating the next

order. Otherwise, someone with a $1000 credit limit could place 100

$990 orders in one day and get them all to ship. In companies where

backorders are common, like publishing, it is also common to want

to have visibility for total orders, even if they aren't ready to

ship and are not involved in credit authorization. Do you want to

query the database every time this is asked? E.g., take my former

customer SYBEX who would get back orders for a future publication

stacked up for every Barnes and Noble store in North America (plus

freight forward orders for others) ... not infrequently multiple

orders per store when they decided to increase the initial

stocking, and then bang, the book is in press and all those orders

are slated for release in the same day ... thousands of them.

Well, I'm talking about a Customer class instance

being mapped to a customer database row. sounded

a lot like it. I might add that some of the

properties of the object might not even be stored

values.

That's the whole point of "transparant persistence": the mapping schema determines which attributes will be persisted in which tables. It's similar to temp-tables and real database tables: the ABL has provided an immense powerfull mechanism by treating temp-tables as database tables. They could do the same thing for classes. Other languages need some complex Object Query Languages, but the ABL can solve it in the runtime. Have a look at DLinq in .Net (http://msdn2.microsoft.com/en-us/library/aa697427(VS.80).aspx), which looks very much like the ABL with its buffers, but it uses objects instead.

E.g., one could elect not to store a total

for the order, but compute one while building the

object.

Sure, no problem!

I'm not sure of your point here. Mine is that tying

the object to the database is unreasonable and

inappropriate due to the likelihood of them not being

on the same machine.

No, there is a runtime mechanism that knows how to load a class from the database. But in a tiered and distributed environment, you still have to cut the direct link between the two tiers: the database mapped object will be transmitted as XML/TempTable/ProDataSet to the client, so the data can be bound to the user interface. The client might be a .Net client that can't consume ABL-classs instances. We shouldn't make the same mistake as EJB, where domain objects are accessed remotely, so every property accessor means a network roundtrip. No, you do a request, remotely something happens to domain objects and persistence and the result is marshalled back as a message.

Admittedly, if the ABL had an envelope which included

multiple of what are now independent sessions, then

one could imagine a pub-sub connection in which a

service would both request the data for an object and

simultaneously register for a change notice.

There is no need to inform the whole world about updates. The paper you referenced is about disconnected, versioned data. Its about the problems that are introduced by that in the new and loosely coupled world.

Given

that the source of the data was the authoritative

source, then it could know that there was a change to

a subscribed record and publish a need to refresh

message or just publish the changes.

This all assumes that the client is able to respond to these published messages. What if I'm a mobile device and I just want to display my lates order. I don't want to be connected to some server all the time....

That is somewhat my reaction as well, but the counter

argument is that temp-tables provide some very useful

services like the to/from XML methods and the

before-table functionality for determining what has

changed.

These are very simple to implement using reflection, runtime inspection of objects. Hopefully this feature will be in the next OOABL release. But even without that, you can generate some code that will cover this issue, don't you think? Mapping data is relatively easy with all the tools available.

And the before-table isn't that usefull in all cases as well. You have to treat it in a special way and in that way it's good. When you need something else, you're in troubles. In the early days of the ProDataSet we kind of fought the before-table, since it didn't want to do what seemed logical.

In fact, I had myself thinking recently about how

many of the languages I have worked in have had some

interesting way of dealing with problems that is not

found in a lot of other languages, but which one

sometimes wants to imitate. And, in specific, how

COBOL was interesting for its ability to define the

same data area in multiple ways.

Data sections are great! I have asked for such a feature in ABL as well. Give us a declarative way of defining data sources. But I think it was you who replied that defining a prodataset was simple enough.

Suppose, for

example, that one could define properties for a

class, but that one could also define a buffer which

contained those properties and the buffer had XML

methods and before-table functionality. Wouldn't

that be neat?

Wouldn't it be nice if you could define class properties and declaratively map them to table columns? And when you would do a "FIND cust" the runtime would recognize this as:

- find "customer" buffer

- create "Customer" class instance

- bind the buffer to the "Customer"

- decouple automatic transactions in this concept

- "SAVE cust" would actually commit the changes to the database

This is what we suggested earlier in this thread....

Do you want to query the database every time this is

asked?

Do you want to update the database everytime something happens to the order? Besides that, certain amounts are related to a date (due payments). And what will happen when you re-base the currency? Do you refresh all totals as well?

I

certainly wanted this when I was first trying to create a

temp-table of Customer objects with a ProDataSet, but once I

figured out how to do that, it wasn't bad and it was certainly far

more flexible than I can imagine a declarative structure being.

I.e., it is the kind of thing that, by the time you make all the

declarations, it takes up about as much effort and space as it does

to make the assignments, especially with the capabilities in 10.1B

where one can have multiple constructors, including one that sets

all properties at once. This seems to me to be a different issue

than I was trying to raise in this thread, however.

I want to update only when

there are meaningful state changes and then only as a part of the

completion process. I certainly wouldn't update this for every

order line change, for example. I haven't seen any credit

authorization filters where the fact that an amount was due in 30

days or 60 days was part of the criteria. I have done credit

authorization that included other factors, such as the age of the

oldest open invoice relative to its due date, but that relies

solely on AR data. FASB-52 tells us how to handle multiple

currencies. It isn't willy-nilly changes. But, hey, if this is a

real problem in your application, feel free to design it so that

these fields are tables with one value per currency instead and

then do the sum dynamically at the time of credit authorization per

current values if that is what you want to do. I don't mind ... it

doesn't really change anything.

Of course, if one is working in a single session against a database, that customer will be in the database buffer for 989 of those reads, so it won't actually be very expensive. Hard to know how you would structure this even in simple, monolithic ABL single session since one certainly wouldn't expect most of the orders to come from one customer.

Ahh - but your missing the cost of building and transmitting 990 Client objects ...

One returns what one is asked for... assuming that the requestor is authorized.

True - but one often doesn't know what they want (and neither frequently does the client).

While I hear your wish, I think it is misplaced. If the database is local, we already get this service performed for us by the database buffer. The real issue is what happens when the authoritative database is not local. This is where I think we have a need for DataXtend type replication, versioned copies, and/or some form of caching come into play.

OO databases do this fine now - why can't the ABL. If we stop thinking about rows at all and only think about objects then there you have it. Why should I even care about the row any more???

Do you want to update the database everytime something happens to the order? Besides that, certain amounts are related to a date (due payments). And what will happen when you re-base the currency? Do you refresh all totals as well?

I want to update only when there are meaningful state changes and then only as a part of the completion process. I certainly wouldn't update this for every order line change, for example.

Actually I'm wondering if we don't need something like a Hibernate or TopLink to handle the "buffer pool" of available objects (this gets back tot he DB handling "objects" linked to tables). If I can as for a "customer object" and get a pointer to the in memory version then I'm set.

Of course, all this is quite complex and I feel quite sorry for the poor PSC C/C++ programmer whose task it is to make our dreams real

In the current state of the art, it appears

that there is often an unacceptable performance penalty for moving

to OO database, AFAIK. And, one certainly complicates the picture

for reporting tools. But, this isn't to say that there might not be

a role for an OO database locally to serve as a cache, even a

shared cache used by multiple sessions. I asked here

http://www.psdn.com/library/thread.jspa?threadID=2918&tstart=0

whether ObjectStore could be used for this purpose, but I haven't

exactly gotten a significant response ... or even an insignificant

one!

I've put in a point back to here since we've had a very good discussion

Actually, I would miss building and transmitting all 1000 Customer objects since I would put the needed Customer information inside the Order object and I would pull the Customer info needed from an Order service cache.

So - that's "lazy loading" - yes? I.e. only loading them when required?

Whether or not it is an

actual OO database, which certainly sounds interesting because it

avoids all the marshalling and demarshalling cost, I definitely

think there is a role for a system cache which is shared by

multiple sessions and possibly by multiple services. This allows

the services to more easily be designed as stateless so that

multiple, replicated copies of a service could all access the same

pool. Three may also be a role for a local copy of a regular

database containing some of the shared data which might be

authoritatively hosted on another machine. When DataXtend supports

Progress (RSN), it could maintain this shared data in near real

time.

Yeah - what is the story with DataXtend ?? I hadn't heard of it until a month or so ago? (Maybe I'm not looking in the right place)

Not necessarily. Imagine an Order Service. It is

connected to a database which houses Order data. This includes

things like unique identifiers of the customer and probably include

pointers to things like bill to and ship to addresses. This Order

Service is also connected to a cache database which houses

information that comes from non-local services. So, it goes to

build the first Order, looks in the cache database for the Customer

details, doesn't find it and a message gets sent to AR to obtain

this Customer View. That data is then put in the cache database and

used to build the first Order object. Then it comes to the second

Order object and does the same thing, but this time it is the same

customer and the data is already in the cache. This isn't really

lazy load. Lazy load might be something more like having the option

during order processing to access a bunch of extended data about an

item, information which is normally not needed. One might elect to

find that information only when needed and that would be a lazy

load.

It was heavily

touted at last year's Exchange and, at the time, they were saying

it would be ready for use with 10.1B, but my understanding is that

it needed a bit more work under the covers than was realized at the

time and that it should support Progress with the next release.

This is very cool technology, not only for this kind of

distributing data around the network so that there are local

copies, but also for handling the knotty problem of the mostly

connected system that is not always available or the problem of the

client that is only rarely connected.

When DataXtend supports Progress (RSN), it

could maintain this shared data in near real time.

I would rather see the OR-mapping and caching being integrated in the runtime for ABL class mapping (http://www.progress.com/dataxtend/or_mapping/index.ssp). And Murray, there is also a C++ version, so maybe PSC is able to pull a trick or two and integrate it more easily (yeah, sure

When DataXtend supports Progress (RSN), it

could maintain this shared data in near real time.

I would rather see the OR-mapping and caching being

integrated in the runtime for ABL class mapping

(http://www.progress.com/dataxtend/or_mapping/index.ss

p). And Murray, there is also a C++ version, so maybe

PSC is able to pull a trick or two and integrate it

more easily (yeah, sure

I'm still waiting for the ABL to fully run in the JVM - that would solve LOTS of issues

I see no reason to think that implementing the AVM within the JVM was on anyones radar.

I see no reason to think that implementing the AVM

within the JVM was on anyones radar.

If it isn't, it should be!!! We get the best of both worlds and PSC still runs everywhere on JVMs. Maybe we could also get PSC for Java Micro Ed so we can run apps on Phones ???

I noted that JRuby is out now: http://www.javaworld.com/javaworld/jw-02-2007/jw-02-jruby.html

I'm quite sure it isn't and I

can think of a lot of reasons why it shouldn't be. Why should PSC

align themselves with Java instead of .NET? Do you actually think

you would get the same performance out of a AVM layered on a JVM?

And, frankly, if one is going to make that drastic a change,

wouldn't one do something different than implementing the current

model? I mean, would you create a single-threaded AVM on top of a

JVM? Wouldn't you want to move all the UI into class libraries?

Wouldn't you in fact want to make the whole thing OO? And, if you

did all of this it would no longer be ABL. Now, that would make a

lot of existing users and APs happy, wouldn't it!

I'm quite sure it isn't and I can think of a lot of

reasons why it shouldn't be. Why should PSC align

themselves with Java instead of .NET? Do you

actually think you would get the same performance out

of a AVM layered on a JVM?

Yes I really do think you can get the same performance from the JVM in Java 5 or especially Java 6. .NET only runs on Windows so its out (lets not get into Mono).

And, frankly, if one is going to make that drastic a

change, wouldn't one do something different than

implementing the current model? I mean, would you

create a single-threaded AVM on top of a JVM?

Wouldn't you want to move all the UI into class

libraries? Wouldn't you in fact want to make the

whole thing OO? And, if you did all of this it

would no longer be ABL. Now, that would make a lot

of existing users and APs happy, wouldn't it!

It can still be the ABL on Java. And yes I'd add lots of new stuff and it could remain single threaded at the start. I'd also look at using an open source AppServer for Java instead of the Progress one (since its now in JVM). We need to look at the where the ABL will be in 10 years, if I'm a betting man, PSC won't still be spending money maintaining their own Virtual Machine - why would you??

Of course - the DB would probably have to remain C/C++ for now, but who knows ...

Yes, I

expect they will ... and I think it is probably the right choice.

You seem to think that Java has won the language wars and that

there is nothing else.

Yes, I expect they will ... and I think it is

probably the right choice. You seem to think that

Java has won the language wars and that there is

nothing else.

No - i think you are confusing the VM with Java the language. They are not the same beast. The VM is a multi-language VM.

That VM hasn't won the VM war any more than Java has won the language war.

That VM hasn't won the VM war any more than Java has

won the language war.

No - but I don't see another good alternative

Nor do I see a need for a universal VM.

No - but I don't see another good alternative

I would imagine the people working on the AVM are following other VM efforts out there and have applied some of the ideas to the language.

At least, I would hope so.

Of course, one might also point out that PSC has been doing their VM a lot longer than most folks have been doing theirs ...

Of course, one might also point out that PSC has been

doing their VM a lot longer than most folks have been

doing theirs ...

Yep - and thats probably why its still single threaded

But also why it is so performant in key areas.

I would imagine the people working on the AVM are

following other VM efforts out there and have

applied some of the ideas to the language.

I think we started talking about PVM (Progress Virtual Machine) on the Directions forum and that has been picked up by marketing guys It doesn't mean that the 4GL runtime is a virtual machine. "Basic" is probably a virtual machine as well in that case...

Perhaps we should go back to the original discussion, where we don't have:

- "transparant persistence" (or ORM)

- multi threading

- generics

The things we do have are buffers, temp-tables, prodatasets, datasources and 4GL database access, hurray!!!

It doesn't mean that the 4GL runtime is a virtual

machine. "Basic" is probably a virtual machine as well in that

case... I would be more than happy to go back to the

original topic of this thread ... which wasn't on any of those

topics.

Actually, looking at the current 10.1B manuals, they are full of references to the AVM.

Actually, looking at the current 10.1B manuals, they

are full of references to the AVM.

It's a matter of repetition like in commercials: at one point you start believing it yourself

And what part of this http://en.wikipedia.org/wiki/Virtual_machine do you feel the AVM doesn't qualify?