Everything dynamic

Posted by Torben on 30-Nov-2014 04:39

Hi,


we need to convert our current system all written in ABL to new ABL backend (hopefully only slightly re-written) and with a brand new web based frontend. (Probably written in some sort of JavaScript framework)
There is some wishes (requirement) that the interface layer between the frontend and backend must be based on REST.

Our current system is based on designs and other data saved in repository (that our customers can customize themselves after some rules defined by us).
These customer changes include the possibility to hide or view fields, set fields required, change size and location of field, add fields from related tables (read-only) or create their own fields that will be handle as if they were part of the ‘table‘ they are added to (full update logic) + more.
Today we basically only have one client program and one server program that handles CRUD operations for any table (entity) in our database. This single program also handles some standard events (like calling programs to view or edit related data entities).
At runtime the server reads repository, database schema, customer created fields, current system configurations and user configurations and from this the server creates ‘modified design file’ that is sent to client and used by client to create windows, frames, fill-in, toggle-boxes and other ABL widgets and a few ActiveX components all dynamically from design sent from server. Server also creates schema for data to be send to from client, matching the current ‘modified design’ with data types, formats, read-only, initial values etc + some hidden control fields.
Beside this single server and client program we have one or more system designs per table (one per entity type) + a program per table (with any special event processing needed for all entities based on that table)
We communicates between client and server with dynamic dataset and dynamic temp-tables. And each AppServer process is bound to a single ABL client, state-reset. (To have the same understanding of schema of the data being send to/from in both client and server)

Now we need to re-do this for WEB frontend and so far I’ve been looking at some of the OpenEdge technologies like BusinessEntity and JSDO, but they all seem to be static in nature!

Are there anyone that has implemented something that is much more dynamic than BusinessEntity and JSDO, that will share what technologies they have and used implementation?

Or alternative are there anyone that is working on making BusinessEntity and JSDO dynamic?

Or share some ideas on how to best do this?

Regards,
Torben

All Replies

Posted by Bill Wood on 30-Nov-2014 06:48

For the record, the JSDO is dynamic. The JavaScript objects are created entirely on-the-fly from the schema defined in the Catalog.

The backend from Progress and Progress Developer Studio for OpenEdge defaults to the statically configured Business Entity and the REST Adapter -- but that is not required by the JSDO.

I know of at least one customer generating the REST (and the Catalog) dynamically. But I will let them respond. =

Posted by bronco on 01-Dec-2014 01:05

I gave a presentation on the EMEA PUG Challenge on a dynamic approach for the REST services, together with code. If you're interested, write me a mail (mail address is listed in my profile).

Posted by Peter van Dam on 07-Dec-2014 09:44

Hi Torben,

Have you made any progress on this question? We have looked into it as well and so far have not found a solution.

Regards,

-peter

Posted by agent_008_nl on 09-Dec-2014 00:49

Could be interesting for more of us. Can't you put your project f.e. on github? Did you use the jsdo in your presentation?

Posted by bronco on 09-Dec-2014 02:12

My presentation can be downloaded at:

www.pugchallenge.eu/.../bronco_oostermeyer_REST%20assured.pptx

the code can be found at either

bitbucket.org/.../pug2014  (Mercurial development repo)

or

https://github.com/bfv/pug2014 (Git snapshot)

This is about a generic approach to the REST adapter, so no JSDO involved. The issue with the JSDO is that when your application landscape is a little bit more complicated than just a HTML5/JS/CSS client and a server, relying on the logic in the JSDO might imply that you have to mimic the functionality of the JSDO in another technology (Java for example). The code only serves as an example for my presentation and undoubtedly lacks in a few areas, error handling to name one.

edited: typos

Posted by agent_008_nl on 09-Dec-2014 07:56

Thanks Bronco. I do not get what you mean with your issue on the JSDO. Can you give an example of a problem in a bit more detail, understandable for a webdummy like me?  Can anyone else deeper into JSDO / REST etc. elaborate on the pros and cons of using the JSDO? On the pros I read:

From www.progress.com/.../Track%201%20-%20Progress%20OpenEdge%20Mobile%20and%20Bits%20Involved.pdf:

REST Hurdles:

It is possible to access data on a server prior to OpenEdge 11.2

•For example using JavaScript XMLHttpRequest( )

•Very labor intensive to write and maintain low level code

Functionality offered by JSDO:

Support for tracking changes

Transparent access to OpenEdge REST web services

Implementation details are hidden

Developers do not need to know the URIs

Local Storage and Offline transport support

Change tracking

Using generic REST Service for development in Mobile App builder will leave out JSDO features.

From oemobiledemo.progress.com/.../index.html

“instead of a Mobile App having to identify the URI, send an HTTP request, and interpret the HTTP response for a given REST resource (AppServer routine call), it only has to call the appropriate JavaScript method or methods on the JSDO to execute the corresponding ABL routine in the class or procedure object on the AppServer. All input parameters, output parameters, and other results of ABL routines are exchanged with the JSDO and its JavaScript methods as corresponding properties of JavaScript objects that the Mobile App passes and returns from the JSDO and its methods.”

“a Mobile service supports some or all of the following Mobile operations that access data and business logic on the AppServer over the network:

• Create — A single operation that creates a single new record on the AppServer.

• Read — A single operation that reads a set of records from the AppServer.

• Update — A single operation that modifies a single existing record on the AppServer.

• Delete — A single operation that deletes a single existing record on the AppServer.

• Submit — A single operation that can perform one or more create, update, and delete operations

in a single transaction on the AppServer. A Mobile resource supports this operation only if you

define it to access a ProDataSet with temp-tables that use before-imaging."

--

Kind regards,

Stefan Houtzager

Houtzager ICT consultancy & development

www.linkedin.com/in/stefanhoutzager

Posted by egarcia on 09-Dec-2014 14:59

Hello,

Just some few comments.

As Bill Wood mentioned above, the JSDO is dynamic and uses the JSDO Catalog to determine the schema and operations supported by the resources.

The JSDO supports invoke operations by name, you just call the method in the JSDO. You do not need to say something like "invoke(methodName)".

REST is useful when you need to have full control of the parameters and headers passed and received from a REST Service.

However, if you need to do special handling of the data, then you need to write it yourself.

The JSDO provides an API that you can use the write code in JavaScript. The JSDO gives you access to features such the JSDO memory (records are available locally) and Submit/before-image support (before-image is included in DataSets so that the Business Entity can detect concurrent changes).

(The JSDO API could also be made available to a Java client or a Swift client.)

Even though the Business Entities generated using Progress Developer Studio are static, it is possible to have a Business Entity that is dynamic to provide CRUD operations for every table in a database. It could have model to allow custom code.

A Mobile Service with a Business Entity is in fact a REST Service where the Business Entity supports a specific set of methods to perform CRUD operations and INVOKE. These methods use a prescriptive approach.

If I had some time available, I would make a generic Business Entity using the sample program customer_rest.p as a starting point.

You can find the sample program customer_rest.p in the middle of the following thread:

community.progress.com/.../47148.aspx

I hope this helps.

Posted by agent_008_nl on 10-Dec-2014 02:24

> (The JSDO API could also be made available to a Java client or a Swift client.)

Servoy provides such a java client if memory serves me right and research is done at the moment to make use of the jsdo. Might be that Peter van Dam knows more of this. ;-)

--

Kind regards,

Stefan Houtzager

Houtzager ICT consultancy & development

www.linkedin.com/in/stefanhoutzager

Posted by egarcia on 10-Dec-2014 06:05

Hello,

Just some few comments.

As Bill Wood mentioned above, the JSDO is dynamic and uses the JSDO Catalog to determine the schema and operations supported by the resources.

The JSDO supports invoke operations by name, you just call the method in the JSDO. You do not need to say something like "invoke(methodName)".

REST is useful when you need to have full control of the parameters and headers passed and received from a REST Service.

However, if you need to do special handling of the data, then you need to write it yourself.

The JSDO provides an API that you can use the write code in JavaScript. The JSDO gives you access to features such the JSDO memory (records are available locally) and Submit/before-image support (before-image is included in DataSets so that the Business Entity can detect concurrent changes).

(The JSDO API could also be made available to a Java client or a Swift client.)

Even though the Business Entities generated using Progress Developer Studio are static, it is possible to have a Business Entity that is dynamic to provide CRUD operations for every table in a database. It could have model to allow custom code.

A Mobile Service with a Business Entity is in fact a REST Service where the Business Entity supports a specific set of methods to perform CRUD operations and INVOKE. These methods use a prescriptive approach.

If I had some time available, I would make a generic Business Entity using the sample program customer_rest.p as a starting point.

You can find the sample program customer_rest.p in the middle of the following thread:

community.progress.com/.../47148.aspx

I hope this helps.

Posted by agent_008_nl on 10-Dec-2014 07:16

Hi Bill,

> I know of at least one customer generating the REST (and the Catalog) dynamically. But I will let them respond.

No response yet? I would be interested to know this too.

--

Kind regards,

Stefan Houtzager

Houtzager ICT consultancy & development

www.linkedin.com/in/stefanhoutzager

Posted by blichal on 10-Dec-2014 07:32

i'm not sure if this is exactly what you're looking for but i recently had to write my own web gui/desktop framework for writing crud applications (based on extjs). we recently went to production with the new framework but we're only using it for read only queries at the moment.

the ui is 100% javascript making rest calls to an openedge appserver. it's not repository based but things like fields formats, validations etc. are stored in a single place in the data object and are created dynamically in grids, forms etc. at runtime. it's similar in concept to smart objects with data objects, grids, forms, modular components etc. but much less complicated.

the project is called breadjs - short for browse, edit, add and delete (crudjs.com was already taken). it's open source. i'll upload the sources and some docs in a couple of month but if anyone is interested, i'll be happy to send him some pictures and code samples.

Posted by blichal on 10-Dec-2014 07:38

btw i think it's very possible to do create a rest adapter with webspeed and progress version 9.

if you write a program that can convert temp-tables to json and back, that's all you need.

there's lots of c/c++ libraries that you might be able to wrap in progress 4gl (i've used one in docxfactory).

Posted by Bill Wood on 10-Dec-2014 08:19

WRT

>>  I know of at least one customer generating the REST (and the Catalog) dynamically. But I will let them respond.

I do want to be respectful of emails and private items, but you can search on Google for "Github JSDO"....  

There are projects (some done internally by progress on)

  • Using Node.js to CALL into OpenEdge (https://github.com/progress/angular-jsdo-demo)
  • Implementing a Data Service in node.js(https://github.com/progress/jsdo-node)  that exposes itself as the service provider for the JSDO (and the Catalog)
  • Implementing a Data Service in Java (https://github.com/gmt-europe/JSDOServer)  that exposes itself as a JSDO (and the Catalog)  

Posted by agent_008_nl on 10-Dec-2014 09:18

Thanks Bill,

When I click on Data Service in node.js or Data Service in Java I get community.progress.com/.../default.aspx (group not found). Is "Using Node.js to CALL into OpenEdge" one of the github projects?

--

Kind regards,

Stefan Houtzager

Houtzager ICT consultancy & development

www.linkedin.com/in/stefanhoutzager

Posted by Bill Wood on 10-Dec-2014 09:27

WRT

>>> When I click on Data Service in node.js or Data Service in Java I get community.progress.com/.../default.aspx (group not found).

Sorry.  Something happened in the different posts to community (I am sure it was user error).  I will update the original post, and include the specific links below as clear text.   (You can also just Google "github JSDO".  That is what I did, and followed the links:

Links I got to:

- https://github.com/progress/jsdo-node

- https://github.com/progress/angular-jsdo-demo

- https://github.com/gmt-europe/JSDOServer

- https://github.com/gmt-europe/JSDOServer/wiki/JSDO-Catalog

There are others as well.

Posted by Torben on 11-Dec-2014 05:57

Hi Peter,

unfortunately 'every day' tasks took precedence, so have not had time to investigate further.

But we still need to get this implemented within the next 1 - 1½ years

/Torben

Posted by pedromarcerodriguez on 11-Dec-2014 17:57

Hi Torben,

We have taken an approach to implement REST that diverges from the official route, but it might be a good fit for your purpose.

Basically we have decided to take an approach based on node.js + MQ + ABL code (without even the requirement for and Appserver).

The idea is we use nodejs (with restify) to define all REST resources and its methods available in our API (in fact this is generated dynamically from a swagger REST api document). Nodejs listen for those requests and deals with some of the middleware responsibility like first level of authentication, CORS, etc... Once that is passed it pushes a message to a MQ server (we use STOMP to connect the nodejs application to the MQ server so we can use almost any option available) with a JSON that includes all parameters corresponding to the request (headers, path, query parameters...).

We also have some ABL listeners connected to the same MQ server (also using STOMP) and listening in that queue, once the message is received, there is a server program that executes all the logic required (this one would be very similar to the one you already have) and sends back to the queue the generated response.

Then that response is passed back from the queue to the nodejs server that sends finally its response back to the client (mobile, web, ...).

Hope that helps and introduces new ideas about how to approach this project.

Regards,

This thread is closed