Practical Implementation - A worked example

Well, first off, I'd have to say that doing almost anything in the domain of addresses is likely to be a problematic area because there are **so** many different ways of structuring addresses. But, with this caveat in mind, I will try a few questions and observations ... questions because I am sure that what you are doing is clearer to you than to me.

State

|-- id (Long) --> Primary Unique

|-- name (String)

+-- (foreign key to country)

State

|-- id (Long) --> Primary Unique

|-- name (String)

|-- zip (String)

+-- (foreign key to state)

I am not sure why you have two versions of "state" here and whether or not "state" is the right word. The only apparent difference is that one has a postal code and the other doesn't. I seem to recall that there are countries which have postal codes, but no states or provinces as well. But, as a general case, I would think it unusual for an entire state or province to have a single postal code, so I'm wondering if this attribute is in the right place.

Now in Australia we can't say name or zip are unique because we can have something like this:

And, in the USA we have cases where a building is physically in one city, but mail is delivered to that part of that city from an adjacent city so it has the postal code of the other city. Not to mention buildings that have their own postal code.

I think that the best that one can really say is that an Address, which doesn't seem to be a part of your model, has a state/province attribute and a postal code attribute and that both can be null according to the individual country. Moreover, where postal codes exist, they may have some hierarchical properties, but these are not consistent country to country and it is not safe to make any assumptions about the relationships of either the geographical units of state/province or city/suburb/district/zone/municipality versus postal codes except that the relationship needs to allow for many to many.

So we will also have 3 (or more) value Objects (VO).

As you can tell from the above comments, I would almost certainly decompose this differently, either will no connection between the geographical and postal attributes at all or, at most, a pointer from the postal code to the geographical unit most typically associated with it. I don't know if other countries do this, but the USA does and it seems like a sensible thing. This is why telephone order people ask for the zip and then verify the city and state ... it is fast to type in and much of the time will produce the correct city.

/You could argue that these go on the state service/

Probably, also the remove.

OO - its easy to do the StateVO (where it includes country) in OO. But how do we handle the "getAllCountries" method?? In Java we return a collection.

Provisionally, I have been thinking in terms of providing a Finder and a DAO. The Finder is responsible for providing a number of different query methods and the DAO handles the actual fetch, whether from the DB or from a service via XML. Thus, one could have multiple active Finders and one DAO. Provisionally, I am thinking in terms of having two types of calls n the DAO, one of which is responsible for returning a single object. If that object is not available or not uniquely identified, then that is an error. The other method type is responsible for returning a collection, although there may be cases where the collection is empty or has only one member. But, my belief is that any one consumer will either be expecting an individual object or a collection.

The curse of copy and paste .. The 2nd one should be Suburb.

Suburb

|-- id (Long) --> Primary Unique

|-- name (String)

|-- zip (String)

+-- (foreign key to state)

"there are *so* many different ways of structuring addresses"

Yep but this is a simple example. We could easily pick other tables. I'm more interested in generating some ""concrete" discussion on a real, but simple, example.

Address comes off the model.

Address

|--id (Long)

|--line1 String

|--line2 String

|...blah blah

However, I don't really care what the tables are - its more that we have a base example to start with. We could call them tableA, tableB and tableC if its less confusing.

SOOOOO

In summary. How about we agree an "example" set of objects, with services, and lets bash something out.

The curse of copy and paste .. The 2nd one should be Suburb.

I thought that something like that might be the case, but I think the argument about decoupling geographic unit and postal code still applies.

Yep but this is a simple example. We could easily pick other tables. I'm more interested in generating some ""concrete" discussion on a real, but simple, example.

I have trouble working on an example when I think that basic decomposition is wrong because it just keeps glaring at me all the way down the line. Of course, the best way to avoid controversial decompositions is to move to the trivial and then one is in danger of it becoming unreal.

|...blah blah

Of course, this is the interesting part!

In summary. How about we agree an "example" set of objects, with services, and lets bash something out.

I suppose my first question is "what exactly are you expecting us to accomplish?'. I say this because my own approach has been more one of trying to look at various pieces, but in their full richness, and try to determine what that piece of the framework should look like. E.g., I started a project on model data access, but got distracted and need now to get back to it.

I want to be able to see a concrete approach to

1. Drawing UML

2. What it outputs

Muz

PS the "blah bah" should always be a special case

1. Drawing UML

2. What it outputs

Do you understand that part about MDA being multiple transforms, some of which act on the same base information but are focused at producing different components and some of which are successive transforms? I.e., really going from even a simple piece of UML to a set of finished code tends to imply that one has a complete framework in place and one has figured out how one wants to deal with all aspects of the architecture. It also implies defining a particular target platform, including the UI(s).

I.e., there are no simple examples. I've spent a fair amount of time already on the DAO and Finder piece and haven't finished that and all that would give you is the data access classes.

Well I'll disagree a bit. I think you can do a simple example that should handle 80% of the cases. Well see (I could be wrong - it happens often - especially according to my wife)

CountryService

CountryVO getCountryByID(Long id);

CountryVO getCountryByName(String name); /*In the

country case, the name must be unique*/

CountryVO[] getAllCountries(); /*Get all

I think this is one of the interesting parts when layering the application:

- you define public interfaces with certain query methods

- you implement the interfaces as stateless services (AppServer)

- you delegate the work to internal classes

- at the end the real work will be done in a data access component

Now when I have to add a new filter, I have to change at least three layers: the interface, the service and the data access object.

There are people that cheat, by generalizing the filtering: they pass a query-string. And the worst version is passing in a database query string. The latter has been the design decission in the ADM-world.

In object relational mapping frameworks you will see object query languages, which translate an object query string to a database query, and finally the rehydration of a collection of objects that fullfill the query. That's pretty far away from real ABL-world.

Besides the filter, you also have to deal with data batches: version one of the solution might return all rows/objects, but a future version should be able to batch the data. Next you will start adding batching parameters to the interface (batch size, current position, etc).

To me this looks like an area Progress could offer developers some productivity gains when they could solve the issue...

CountryService

Long createCountry(countryVO CountryVO); /*Returns

the new unique ID*/

...

DAO

===

So the straight DAO objects (e.g. State) are easy,

they become DB tables. The checking for foreign keys

could be "delete validation triggers" or something

similar.

When you pass value objects between layers, you're doing a data centric architecture. This means that the data is disconnected from the actual logic:

- the services receives the data and might massage it a bit

- in most cases this same data stream is passed down

- the data access component in most cases accepts the external data stream in the end

There are a couple of problems with this approach:

- the data stream is 9 out of 10 cases designed from the datasource perspective. This means it will almost always resemble the physical database structure (it's not always a problem, since the database reflects the data you want to register of course!)

- you create dependencies between the layers, since the data source object as well as the service object rely on the same data stream structure.

- when you design DAO-classes that map to a single table, the DAO can't load additional data, since it only knows about the table it manages. This type of DAO is relative easy to generate or to make a generic component for it with somekind of database mapping (attribute x maps to database field y). It's value is relatively restricted... unless you put the powerfull filters/queries in there as well. You could also design these queries as standalone classes. But the problem is the return value: in some cases it's more efficient to return the actual query in other cases it might be perfectly sensible to return a temp-table with all data.

There are implementations where the data access class accepts simple value types only, so createCountry in the data access class would be something like:

void createCountry(string countryName, ....., out long countryId);

My point is that, even if the actual code related to the specific small fragment of UML is small, it needs to go in a context. Without the context, one doesn't know what code should be there. Set aside the UML for a moment and think in terms of hand coding the same example. Before you ever write anything about countries, you need security services, context services, etc. Until those are defined, you don't know how they are going to be called. This is the up front cost of building an OERA application ... one needs the framework.

This is why I propose a two layer data access layer, a Finder and a DAO. In 99% of the cases, adding a new filter is simply a question of adding something to the Finder. The DAO remains the same and the only object outside of the Finder that needs to know about the new method is the one that will use it.

When you pass value objects between layers, you're doing a data centric architecture. This means that the data is disconnected from the actual logic:

If you pass the object, then the logic travels with the data. If you are sending a temp-table or XML representation of the object across the wire, then you will form a new object at the other end. Depending on intent, this can be exactly the same object or it can be specialized for the context, e.g., possibly read-only.

the data stream is 9 out of 10 cases designed from the datasource perspective. This means it will almost always resemble the physical database structure

I see absolutely no reason for this. To be sure, design is easiest whent the DB structure closely resembles the object structure, but there is no reason for anything above the DAO to have the slightest clue how the data are stored.

you create dependencies between the layers, since the data source object as well as the service object rely on the same data stream structure.

Don't see any reason for this either. The object structure is the object structure. One can diddle with the database or substitute XML over the wire and nothing above the DAO has the slightest clue.

when you design DAO-classes that map to a single table, the DAO can't load additional data, since it only knows about the table it manages.

Why? E.g., there can be a table containing transaction data which contains codes and the DAO can expand those codes into full descriptions which are a part of the object. Addresses might be stored in a separate table in the database, but be part of the transaction object. Etc., etc.

unless you put the powerfull filters/queries in there as well. You could also design these queries as standalone classes

Hence my recommendation of splitting out the Finder object and keeping the DAO itself very generalized.

But the problem is the return value: in some cases it's more efficient to return the actual query in other cases it might be perfectly sensible to return a temp-table with all data.

As noted elsewhere, I do think that there are times that one expects a single unique record and there are times that one expects a collection (which may have 0, 1, or more members) and one should have methods accordingly. But, I wouldn't return a temp-table, I would return a collection object containing all of the object instances in the collection.

void createCountry(string countryName, ....., out long countryId);

Why are you making the ID a return parameter instead of the result of the method?

BTW, I think that one of the implications of designing for SOA is that objects should have two constructors, one that initializes the object with no or very limited initial data and one the initializes a full blown object with all data based on XML.

I think this is one of the interesting parts when layering the application:

- you define public interfaces with certain query methods

- you implement the interfaces as stateless services (AppServer)

- you delegate the work to internal classes

- at the end the real work will be done in a data access component

Now when I have to add a new filter, I have to change at least three layers: the interface, the service and the data access object.

There are people that cheat, by generalizing the filtering: they pass a query-string. And the worst version is passing in a database query string. The latter has been the design decission in the ADM-world.

The other option of course is to look at what happens in the "real" world where in 80% or more of the cases, the service object IS the same as the DAO (I think you mention this later). Hence, one wonders if you even need a ValueObject (or DTO if you prefer) to pass around. Maybe we should simply pass a DAO ??

>when you design DAO-classes that map to a single table, the DAO can't load additional data, since it only knows about the table it manages.

Yep - thats exacly how I see it working. You can wrap multiple DAOs (single tables) in a higher level structure (maybe a ProDataset) which is where you perform business logic on it. Of course, this is where the ValueObject is supposed to come it. Its the "multiple table" or "businesS" view of the phsyical data.

The ""other"" option is to take Hibernate's approach and allow "lazy loading" and load the tables in the background. Now you could do something like this in the ABL but I'm not convienced that its needed.

This is why I propose a two layer data access layer,

a Finder and a DAO. In 99% of the cases, adding a

new filter is simply a question of adding something

to the Finder. The DAO remains the same and the only

object outside of the Finder that needs to know about

the new method is the one that will use it.

Now in the EJB3 world, the "finder" methods are in the DAO BUT since everything actually loads an "abstract class" you can change the implementation of the DAO without having to re-compile. I look at it as the same as adding an extra index to the DB without changing the CRC and only re-compiling the code you need to use said index.

No, I definitely want two separate objects because I think that one might have multiple currently active Finders for one DAO. See Fowler.

You can wrap multiple DAOs (single tables) in a higher level structure

You seem to be assuming a 1 to 1 relationship between object and table, which I think would often not be the case in practical terms as well as undesirable in theoretical terms. E.g., if one is going to allow ship-to by line in an order, which many people do allow, then are you actually going to store the address in each line or is that address going to be stored somewhere else and only be pointed to by the line. The latter, I hope, so the object will have the addresses within it, but the database will have the addresses separate, even if they are designed hand in hand.

Maybe we should simply pass a DAO ??

Pass a data access layer object to a domain layer object ???

No, I definitely want two separate objects because I

think that one might have multiple currently active

Finders for one DAO. See Fowler.

Yes, I agree. I was thinking that we would simple use a "serviceLocator" to find a reference to an existing or new object that we could query. So we find the DAO object with the "serviceLocator", and do stuff - like this:

stateDao = serviceLocator.getStateDao();

StateVO[] sVO = stateDao.getAllStates();

or, if you want to ditch the VO concept

State s = stateDao.getState(uniqueID);

Not sure if I'm on the same wave length today (sorry - I'm suffering from the end of a migraine).

You can wrap multiple DAOs (single tables) in a

higher level structure

You seem to be assuming a 1 to 1 relationship between

object and table, which I think would often not be

the case in practical terms as well as undesirable in

theoretical terms. E.g., if one is going to allow

ship-to by line in an order, which many people do

allow, then are you actually going to store the

address in each line or is that address going to be

stored somewhere else and only be pointed to by the

line. The latter, I hope, so the object will have

the addresses within it, but the database will have

the addresses separate, even if they are designed

hand in hand.

I think you need both (for speed). If I want to look up 100,000 lines (e.g. to count the lines for a day) then I don't want to load the other objects too. Yes it comes down to good design I know but occasionally mistakes are made.

OR - maybe I want to add 1 to every line on an order line. I don't want to write a special method so I do something like this:

OrderLineVO[] o = orderDao().getLinesForOrder(1);

if (o != null) {

for (i = 0;i

o.setValue(o.getValue() + 1);

orderService.updateOrder(o);

}

}

Now I don't want to load the address too. Hence I will likely need multiple VOs ... OR I will use DAO objects and hide it all in the Business Logic layer.

I don't think we are thinking quite the same. When a business logic layer object wants a new domain object or collection of domain objects, it instantiates a Finder for that object and provides it with the appropriate instructions on what to get and then calls a method to actually perform the get. The Finder either attaches to an existing DAO for that object or instantiates a new one and tickles it to do the fetch and assembly, then passes the result back to the BL layer object. The Finder serves as a sort of interface object to the DA layer.

That's pretty far away from real ABL-world.

Besides the filter, you also have to deal with data batches: version one of the solution > might return all rows/objects, but a future version should be able to batch the data. >

Next you will start adding batching parameters to the interface (batch size, current

position, etc).

To me this looks like an area Progress could offer developers some productivity gains

when they could solve the issue...

Maybe we need to make a list of useful "language enhancements". 4GL --> ABL --> 5GL ???

Anyway - I want annotations (I've already sent details to Bedford on why and how) as in EJB 3. They are really useful when generating WebServices - especially for versioning (but that's a side topic).

Maybe we could do something like:

customer.p

/@DAO - Primary, Unique/

/@Finder - Unique find/

def var i as int no-undo.

/@Finder - findByName [] /

def var name as char no-undo.

/@OneToMany - order (orderId)/

order.p.

/@DAO - Primary, Unique/

/@Finder - Unique find/

/@ManyToOne - customer/

def var orderId as int no-undo.

This could then create the DAOs, DB objects and finder methods all in one.

Yes - I agree

Maybe we need to make a list of useful "language enhancements". 4GL --> ABL --> 5GL ???

To get what I would think of as a 5GL, I think you would have to add a new layer because there is too much baggage in the ABL. But then, perhaps that is the way we should be thinking of UML.

Anyway - I want annotations (I've already sent details to Bedford on why and how) as in EJB 3.

Would you please go to my OO Wish List and add this? http://www.oehive.org/OOWishList I will incorporate it when I fold in the comments.

This could then create the DAOs, DB objects and finder methods all in one.

I think you are going to need a wee bit more than that ... but then it is there in the UML.

BTW, what are those .p extensions doing there?

BTW, what are those .p extensions doing there?

A friendly hint to anyone from PSC reading

We know they are laggards ... doesn't mean we can't set a good example ... indeed, a good example would be very welcome!