Need some input and/or good ideas.
I've got an appserver program that returns our products in a large number of ways depending on input. It's got something like 25 differnt input parameters, CHARS, DATES and INTEGERS.
This program is called from a web front that collect various data from the user and then renders a view of our products. Works like a charm!
Roughly what I've got:
RUN returnProducts.p ON ghServer
( INPUT productTypes , INPUT fromPrice , INPUT toPrice , INPUT fromGrade , INPUT toGrade , INPUT parameter1 , INPUT parameter2 , INPUT parameterN , INPUT sortingOption , INPUT pageToShow , INPUT resultsPerPage , OUTPUT ttFilteredProducts).
Now I want to specify certain fixed parameter settings to use at various points, for instance for background jobs generating xml-feeds with our products as well as webpages returning specific products on sale. Something like 50-100 different such settings will exists. The final goal is for our sale managers to be able to set up these parameters themselves and then make that collection of products into a webpage, xml-feed, rss-feed etc.
What I need is a table in which to store all parameter settings together with some metadata. I can see some options here:
1) Store parameters in a table with one field for each input parameter. Positive: easy to do. Negative: changing parameters will lead to schema changes.
2) Store all parameters in a single character field with some kind of delimiter. For instance first entry in a pipe delimited value is the first parameter, second entry is second parameter etc. Positive: easy and flexible schema . Negative: harder to maintain, require a more complicated UI to make sure data types are correct etc.
3) Rewrite the server-side program to take a value-pair TEMP-TABLE (ie tt.parameterName + tt.parameterValue) as input instead. Positive: easy to translate into a matching db table and flexible. Negative: lots of work to redo server-side program as well as existing UI.
Oh, and not OO!
Any better ideas?
1) Store parameters in a table with one field for each input parameter. Positive: easy to do. Negative: changing parameters will lead to schema changes.
3) Rewrite the server-side program to take a value-pair TEMP-TABLE (ie tt.parameterName + tt.parameterValue) as input instead. Positive: easy to translate into a matching db table and flexible. Negative: lots of work to redo server-side program as well as existing UI.
Need some input and/or good ideas.
I've got an appserver program that returns our products in a large number of ways depending on input. It's got something like 25 differnt input parameters, CHARS, DATES and INTEGERS.
This program is called from a web front that collect various data from the user and then renders a view of our products. Works like a charm!
Rougly:
RUN returnProducts.p ON ghServer
( INPUT productTypes
, INPUT fromPrice
, INPUT toPrice
, INPUT fromGrade
, INPUT toGrade
, INPUT parameter1
, INPUT parameter2
, INPUT parameterN
, INPUT sortingOption
, INPUT pageToShow
, INPUT resultsPerPage
, OUTPUT ttFilteredProducts).
Now I want to specify certain fixed parameter settings to use at various points, for instance for background jobs generating xml-feeds with our products as well as webpages returning specific products on sale. Something like 50-100 different such settings will exists. The final goal is for our sale managers to be able to set up these parameters themselves and then make that collection of products into a webpage, xml-feed, rss-feed etc.
What I need is a table in which to store all parameter settings together with some metadata. I can see some options here:
1) Store parameters in a table with one field for each input parameter. Positive: easy to do. Negative: changing parameters will lead to schema changes.
2) Store all parameters in a single character field with some kind of delimiter. For instance first entry in a pipe delimited value is the first parameter, second entry is second parameter etc. Positive: easy and flexible schema . Negative: harder to maintain, require a more complicated UI to make sure data types are correct etc.
3) Rewrite the server-side program to take a value-pair TEMP-TABLE (ie tt.parameterName + tt.parameterValue) as input instead. Positive: easy to translate into a matching db table and flexible. Negative: lots of work to redo server-side program as well as existing UI.
Any better ideas?
Flag this post as spam/abuse.
Thanks!
Actual values is basically all I need. Metadata would be a bonus but I think both your ideas are good. This will require some thinking...
Need some input and/or good ideas.
I've got an appserver program that returns our products in a large number of ways depending on input. It's got something like 25 differnt input parameters, CHARS, DATES and INTEGERS.
This program is called from a web front that collect various data from the user and then renders a view of our products. Works like a charm!
Rougly:
RUN returnProducts.p ON ghServer
( INPUT productTypes
, INPUT fromPrice
, INPUT toPrice
, INPUT fromGrade
, INPUT toGrade
, INPUT parameter1
, INPUT parameter2
, INPUT parameterN
, INPUT sortingOption
, INPUT pageToShow
, INPUT resultsPerPage
, OUTPUT ttFilteredProducts).
Now I want to specify certain fixed parameter settings to use at various points, for instance for background jobs generating xml-feeds with our products as well as webpages returning specific products on sale. Something like 50-100 different such settings will exists. The final goal is for our sale managers to be able to set up these parameters themselves and then make that collection of products into a webpage, xml-feed, rss-feed etc.
What I need is a table in which to store all parameter settings together with some metadata. I can see some options here:
1) Store parameters in a table with one field for each input parameter. Positive: easy to do. Negative: changing parameters will lead to schema changes.
2) Store all parameters in a single character field with some kind of delimiter. For instance first entry in a pipe delimited value is the first parameter, second entry is second parameter etc. Positive: easy and flexible schema . Negative: harder to maintain, require a more complicated UI to make sure data types are correct etc.
3) Rewrite the server-side program to take a value-pair TEMP-TABLE (ie tt.parameterName + tt.parameterValue) as input instead. Positive: easy to translate into a matching db table and flexible. Negative: lots of work to redo server-side program as well as existing UI.
Any better ideas?
Flag this post as spam/abuse.
And, if you want to make the solution technology neutral, consider passing JSON equivalent to the temp-table instead of the temp-table itself since then any kind of client can submit a request.
And, if you want to make the solution technology neutral, consider passing JSON equivalent to the temp-table instead of the temp-table itself since then any kind of client can submit a request.
Flag this post as spam/abuse.
Well, Matt, I would question a couple of points. It is fine to deal with things as a TT on the AppServer and, at the last minute, use WRITE-JSON to provide a parameter to pass. Likewise, at the ABL client end, a single READ-JSON should turn that back into a TT so there is no need to write JSON handling code at either end. And, a number of the possible consuming clients are not REST, but rather Java, .NET, or Web clients calling to the AppServer.
And, if you want to make the solution technology neutral, consider passing JSON equivalent to the temp-table instead of the temp-table itself since then any kind of client can submit a request.
Flag this post as spam/abuse.
In ABL you don't know the serialization format for temp-tables. You don't care. It just works as the run statement takes care of it for you. The RUN statement in this case is your boundary.
As Mike points out, the open clients, ABL clients, appserver, and REST adapter all deal with it with varying degrees of transparency. So from the ABL application developer perspective the message encoding is a black box and you don't worry deal with it.
I also agree with his point about strong typing. You lose that when you use name-value pairs. BUT...
When straight ABL (ignoring open clients and rest adapter for the moment) you cannot do this with objects. 11.5 introduced ABL object serialization across appserver boundary, but support wasn't added for the open clients and rest adapter, files and so on. So the serialization format isn't technology neutral yet, your application has to deal with it. I speculate that eventually this will be addressed.
If you pick JSON and bundle it into a longchar variable you've chosen your message encoding at the application layer instead of at the transport layer. You can no longer change it. You want XML? That's a new parameter to request XML. Ignore the "efficiency" arguments for the moment. You still haven't transported it yet, you have merely changed the encoding from say "object" to "string".
So for future proof you need a technology (read: transport) neutral solution that doesn't burn the message encoding into the application layer, allows easy additions of new name-value pairs so API signatures don't have to change (read: the .p running on the appserver), and allows for new future encodings.
The choice is based on the technologies and future technologies in play. If you'll never use open client, and always use OOABL (for the moment), then objects are a first place choice. Or you can bet in the future that additional OOABL serialization options become available to support those other transports. If you are using/going open client, or REST adapter, then and you need it now, then this isn't a good choice.
In ABL you don't know the serialization format for temp-tables. You don't care. It just works as the run statement takes care of it for you. The RUN statement in this case is your boundary.
As Mike points out, the open clients, ABL clients, appserver, and REST adapter all deal with it with varying degrees of transparency. So from the ABL application developer perspective the message encoding is a black box and you don't worry deal with it.
I also agree with his point about strong typing. You lose that when you use name-value pairs. BUT...
When straight ABL (ignoring open clients and rest adapter for the moment) you cannot do this with objects. 11.5 introduced ABL object serialization across appserver boundary, but support wasn't added for the open clients and rest adapter, files and so on. So the serialization format isn't technology neutral yet, your application has to deal with it. I speculate that eventually this will be addressed.
If you pick JSON and bundle it into a longchar variable you've chosen your message encoding at the application layer instead of at the transport layer. You can no longer change it. You want XML? That's a new parameter to request XML. Ignore the "efficiency" arguments for the moment. You still haven't transported it yet, you have merely changed the encoding from say "object" to "string".
So for future proof you need a technology (read: transport) neutral solution that doesn't burn the message encoding into the application layer, allows easy additions of new name-value pairs so API signatures don't have to change (read: the .p running on the appserver), and allows for new future encodings.
The choice is based on the technologies and future technologies in play. If you'll never use open client, and always use OOABL (for the moment), then objects are a first place choice. Or you can bet in the future that additional OOABL serialization options become available to support those other transports. If you are using/going open client, or REST adapter, then and you need it now, then this isn't a good choice.
Flag this post as spam/abuse.
What I dislike about passing TT's across the AppServer boundary is the coupling it creates between subsystems. A TT is a very specific implementation. The JSON corresponding to that implementation is just a message ... a message which seems to be understood by all current technologies in a native way. Put the translation in an interface layer, if you wish.
What I dislike about passing TT's across the AppServer boundary is the coupling it creates between subsystems. A TT is a very specific implementation. The JSON corresponding to that implementation is just a message ... a message which seems to be understood by all current technologies in a native way. Put the translation in an interface layer, if you wish.
Flag this post as spam/abuse.
Because a TT is a very specific ABL implementation whose definition has to be shared by both ends of the communication. A JSON string is just a string.
Naturally, both ends have to agree on message format or meaning or the message would make no sense. The point is that a JSON string is physically just a string and the interpretation of that string makes no assumption about technology, structures, anything. Whereas, a TT is a very specific ABL implementation artifact which has to be translated into a very specific implementation artifact in the target language. That is strong coupling at the level of implementation.
That it works is not the point. If the only possible client was an ABL client, I would still object because of the need to have matching TT definitions. The TT definition in an include file thing is a giveaway to me of implementation coupling.
The point is that a JSON string is physically just a string and the interpretation of that string makes no assumption about technology, structures, anything.
-- Tom Bascom 603 396 4886 tom@greenfieldtech.com
Interesting idea Thomas, does this mean you just don't pass temp tables datasets to appsrv even when connecting from abl client and prefer instead to send data serialized as JSON?
The distinction is that a TT is an ABL-specific implementation. JSON is a standard message format understood by a wide range of technologies. A TT is not just data, but data and behavior. A JSON string is just data. That sender and receiver have to agree on the meaning of the message is tautological. But, for example, a receiver can choose to pluck out a single data item from the message and pay attention only to that, but it has to have an entire counterpart to the TT.
@Marian ... it has been a long time since I wrote any client code or even application code, so the issue hasn't arisen, but I would certainly be thinking in those terms, yes.
You don't see the difference between an ABL-specific implementation that has behavior like indexes and the like and a simple string written in a language independent format?
Not with any practical relevance!
Indexes are optional on temp-tables.
It's all about fit to purpose.
If I've got an ordered set of data a temp-table with appropriate
indexes is a pretty darned good fit.
Even if it isn't ordered -- a set of repeating rows is very good fit
with a TT.
JSON is obviously a better fit for "one off" or "every message is
different" kinds of applications.
JSON is not such an obviously good fit for table-like purposes.
Kind of like Mongo is a good database for not very structured data
-- but you probably don't want to use it for your payroll system.
RE:
Storing input parameters
Reply by Thomas Mercer-HurshYou don't see the difference between an ABL-specific implementation that has behavior like indexes and the like and a simple string written in a language independent format?
Stop receiving emails on this subject.Flag this post as spam/abuse.
-- Tom Bascom 603 396 4886 tom@greenfieldtech.com
TT have an default index even if one doesn't define one. The data types are ABL specific. They are just miles away from a technology independent message. The only reason they work as parameters with non-ABL clients is because PSC has chosen to implement a map from the ABL specific implementation to an implementation specific to the other language.
You don't see the difference between an ABL-specific implementation that has behavior like indexes and the like and a simple string written in a language independent format?
Flag this post as spam/abuse.
--
Tom Bascom
603 396 4886
tom@greenfieldtech.com
Flag this post as spam/abuse.
You don't see the difference between an ABL-specific implementation that has behavior like indexes and the like and a simple string written in a language independent format?
Flag this post as spam/abuse.
--
Tom Bascom
603 396 4886
tom@greenfieldtech.com
Flag this post as spam/abuse.
Flag this post as spam/abuse.
If I'm talking to the outside world sure. If I'm talking to OE?
Don't be silly.
It's still awfully wordy. Not as bad as XML but bad enough.
If the OE database stored JSON in an indexable and queryable manner
that would be really cool. Stuffing it into a CLOB where you cannot
do much of anything useful with it is just silly.
RE:
Storing input parameters
Reply by Peter JudgeOne big benefit of JSON is that just about anyone talks it (similar to XML). That benefit outweighs its weak-typeness, most of the time.
-- Tom Bascom 603 396 4886 tom@greenfieldtech.com
Wordy? Have you compared the data packet size of a TT vs the corresponding JSON?
Have you? Didn't you mention above that you don't write any code?
Getting the size of the JSON is easy enough -- I'd be kind of
curious how one might determine the TT size.
RE:
Storing input parameters
Reply by Thomas Mercer-HurshWordy? Have you compared the data packet size of a TT vs the corresponding JSON?
-- Tom Bascom 603 396 4886 tom@greenfieldtech.com
That's why one would have to look at the actual packet. I am remembering Greg's experiments way back when when he discovered that he could convert to XML and send that and put it back in a TT faster than he could send the TT. That was before the options to not send schema and stuff, but it still makes me wonder.
Oh, and I said little "client" code, that doesn't mean no code.
That's why one would have to look at the actual packet. I am remembering Greg's experiments way back when when he discovered that he could convert to XML and send that and put it back in a TT faster than he could send the TT. That was before the options to not send schema and stuff, but it still makes me wonder.
Flag this post as spam/abuse.
I don't have a quick and easy way to take a peek at the network packets
all deployed and ready to go with test code for this purpose. But I
expect that it is reasonable to think that they are not significantly
bigger than the in memory size of the TT.
I do just happen to have some code that turns a non-trivial ProDataSet
into JSON. And just for kicks and grins I added a little loop to add up
the buffer:record-length as it wanders through the dataset. That only
took a minute or two.
In this case the JSON is about 3x the size of the TT/PDS.
Probably mostly because the TT isn't repeating the field names over and
over and over and over with each record. There is other useless junk in
the JSON but the bulk of the wordiness is from repeating the field names
with each iteration of a record.
So, yes, JSON is "wordy" compared to a Progress TT.
Not as bad as XML though -- if I spit out XML it is usually at least 6x
or 8x the size of the TT/PDS (the field names get repeated *twice* per
record and there is lots more extra decoration thrown in).
I would expect the results to be highly dependent on the nature of the specific TT or PDS and for the wire version to be quite different than the in-core version.
No, I don't expect the developer to do it different ways. I expect them to do it the same way every time.
In particular, if messages passed across subsystem boundaries are ABL dependent technology, then if you change the technology in one of the subsystems you will have to rework the interface. If it isn't, you don't. Perhaps the AppServer boundary in specific includes some magic that allows you to fake the relationship, but that assumes the communication will always be over an AppServer boundary. What happens, for example, when you implement a messaging system to communicate to a remote system ... not AppServer, but some standard messaging protocol, e.g., if the remote system is not ABL. Maybe its COBOL! Then the JSON message will go over any protocol you choose and the TT won't.
I would expect the results to be highly dependent on the nature of the specific TT or PDS and for the wire version to be quite different than the in-core version.
In particular, if messages passed across subsystem boundaries are ABL dependent technology, then if you change the technology in one of the subsystems you will have to rework the interface. If it isn't, you don't. Perhaps the AppServer boundary in specific includes some magic that allows you to fake the relationship, but that assumes the communication will always be over an AppServer boundary. What happens, for example, when you implement a messaging system to communicate to a remote system ... not AppServer, but some standard messaging protocol, e.g., if the remote system is not ABL. Maybe its COBOL! Then the JSON message will go over any protocol you choose and the TT won't.
Flag this post as spam/abuse.
That means if there will be a mean to pass the message in one's own language and the message is correctly delivered to the other end in their native language we should still stick to using English just because we don't want to be dependent on that transport technology?
If I need a temp-table on abl, a result set on java or a dataset on .net then would rather use the opportunity to pass them across if possible as-is and don't care much how it's been done instead of writing my own serialization routine... I'm pretty sure Thomas you were requesting passing objects across appsrv boundaries and you'll probably agree tt's and datasets are still objects (you do mention those are more than data storage but have methods and hence functionality).
The data types are not abl specific, those gets translated just fine in java/.net equivalent while one can argue JSON is just using the JS data types... in fact it is what it is, a JS object notation.
Beside, as Mike said, the serialization (if needed) can be done by a very thin application component that is usually part of the framework so the developer doesn't need to care it's data structure will have to travel in any specific serialization format. The developer works with language specific data types, not every developer need to worry about how those gets serialized if a message need to be sent out or received into the system.
Thank you for ideas (and in depth discussion on pros/cons/philosophy of temp-tables vs json vs xml). I'm currently on 11.4 by the way...
Actually, Marian, I consider passing objects across the wire anathema.