Questions about the OpenEdge COMPILE Statement

Posted by dbeavon on 01-Oct-2019 23:12

I'm working with the COMPILE statement and trying to learn as much as I can about it's behavior.  My company is not in the business of writing and selling OE compiler tools or anything like that...  But unfortunately we can't seem to avoid this level of technical detail...

Q1. If the statement encounters an error, is the _progres client AVM that is executing the statement supposed to crash altogether?  I am observing that it occasionally crashes our AVM during the COMPILE statement, and the error message in the event log seems to be pointing us to a file I/O issue.

(I noticed that there is a COMPILER system handle and it has an ERROR attribute and that seems like it intends to return a more friendly error.  See https://documentation.progress.com/output/ua/OpenEdge_latest/index.html#page/dvoop/using-the-compiler-system-handle.html .  Or better yet, it would be nice if Progress would raise a catch'able ERROR's via SEH ).

Q2. Question from the docs : https://documentation.progress.com/output/ua/OpenEdge_latest/index.html#page/dvref/compile-statement.html

Compiles a procedure file or a class definition file. A compilation can last for a session, 
or you can save it permanently for use in later sessions (
as an r-code file, which has a .r extension).

Is it possible to compile in one operation and then use the SAVE argument in a subsequent operation to save the r-code, but avoid incurring additional CPU overhead during a subsequent COMPILE w/SAVE?  We are attempting to do compilation work in parallel on multiple _progres sessions.  This is in order to make use of our multi-processor hardware.  But the SAVE part of this is a bit tricky because they result in file I/O and concurrency issues.  It would be nice if all the sessions could "safely" do a memory compilation in parallel.  Then , as a secondary step, the results would be synchronized as they were written to disk (which is less safe where concurrent processes are concerned)

Q3.a. Further in the docs:

When you compile a class definition file, ABL compiles the class definition file identified 
in the COMPILE statement and all class files in its inherited class hierarchy, by default.

I'm curious about the "by default".  How do we influence whether the compiled output of multiple (inherited) classes are written or just the single class that was actually the target of the COMPILE request.  It is a problem for our compiler tooling that there are an unknown and unexpected number files being written to disk.  Ideally we would be able to limit the impact on disk, and write only the single class file that was requested.  The remaining r-code files can be written in due course once our compiler tooling is explicitly asking for them.

 

 

Q3.b. If multiple r-code files are going to be unexpectedly written to disk (without being requested) then is there an API for telling us (maybe before or after the fact) *what* particular files are being written to disk?

Q4. I'm confused by the syntax of SAVE where it allows for a "logical-expression". ...

If you specify a logical-expression, its value determines whether the STREAM-IO option is activated.

Is this a way to conditionally avoid saving r-code files in the inheritance tree until they are explicitly requested?

Q5. You will probably be aware by now that I'm not a fan of the saving of r-code files which weren't requested.  It causes problems when making use of available CPU for parallel compilation.  Whoever came up with this approach was obviously not thinking about the potential for concurrency failures. So here is another question with a goal of avoiding the behavior of saving multiple files.  

There is a MULTI-COMPILE attribute on the COMPILER handle.  First I set that to TRUE. Next I would like to compile classes A, B, and C and write them all to disk.  They are inheriting from each other like so : A->B->C.    As a preparation I first compile them all in memory without using the "SAVE" argument.  Now I want to persist the results on disk, since they are presumably stored in cached memory.  If I use "COMPILE" with the "SAVE" argument on class C, then is the expected behavior that it should only write a single r-code to disk for class C?  (Leaving A and B unwritten?)

Q6.  Along the same lines as Q3.b, is there an API we can use to get the compiler to tell us (proactively) what the cls files are that it needs to work with as a dependency before it can compile the one that is ultimately requested?  I suppose this might be considered a form of "reflection", but it would be used in order to request the compilations in the correct sequence (ie. in a sequence that prevents the compiler from writing multiple output files as a result of a single COMPILE statement).

 

Q7a. I understand that Progress now distributes PCT with their OE installations.  Hopefully this will unburden many customers from having to create their own tooling for compiling and packaging their ABL solutions.  Is this fully supported by Progress yet? 

Q7b. Are there any alternatives from Progress for those of us that aren't currently part of the Java/ANT ecosystem?

Any help would be greatly appreciated.  I never guessed we'd end of in the business of developing OE compiler tools...

Posted by Laura Stern on 02-Oct-2019 20:41

Answers to the things I know about, in-line.

Q1. If the statement encounters an error, is the _progres client AVM that is executing the statement supposed to crash altogether?  I am observing that it occasionally crashes our AVM during the COMPILE statement, and the error message in the event log seems to be pointing us to a file I/O issue.

--- No.  Of course the AVM should not crash when it encounters an error during compilation - or ever for that matter.  It should just display an error message.

Q2. ...Is it possible to compile in one operation and then use the SAVE argument in a subsequent operation to save the r-code, but avoid incurring additional CPU overhead during a subsequent COMPILE w/SAVE?  ...

--- No.  When you use the COMPILE statement, it will compile the code even if you had compiled it before and there is an in-memory version of the r-code.

Q3.a. ...When you compile a class definition file, ABL compiles the class definition file identified in the COMPILE statement and all class files in its inherited class hierarchy, by default. I'm curious about the "by default".  How do we influence whether the compiled output of multiple (inherited) classes are written or just the single class that was actually the target of the COMPILE request.  ...

--- When you compile a child class the parent(s) will also get compiled.  If you say SAVE, all levels are saved.  There is no way to make it not do this.

Q3.b. If multiple r-code files are going to be unexpectedly written to disk (without being requested) then is there an API for telling us (maybe before or after the fact) *what* particular files are being written to disk?

--- No

Q5. ... There is a MULTI-COMPILE attribute on the COMPILER handle.  First I set that to TRUE. Next I would like to compile classes A, B, and C and write them all to disk.  They are inheriting from each other like so : A->B->C.    As a preparation I first compile them all in memory without using the "SAVE" argument.  Now I want to persist the results on disk, since they are presumably stored in cached memory.  If I use "COMPILE" with the "SAVE" argument on class C, then is the expected behavior that it should only write a single r-code to disk for class C?  (Leaving A and B unwritten?)

--- I assume C is the bottom of the hierarchy.  No.  Same as the answer to Q2 and Q3.  It will compile all 3 and save all 3 to disk.

Q6.  Along the same lines as Q3.b, is there an API we can use to get the compiler to tell us (proactively) what the cls files are that it needs to work with as a dependency before it can compile the one that is ultimately requested?  I suppose this might be considered a form of "reflection", but it would be used in order to request the compilations in the correct sequence (ie. in a sequence that prevents the compiler from writing multiple output files as a result of a single COMPILE statement).

--- No.

All Replies

Posted by Laura Stern on 02-Oct-2019 20:41

Answers to the things I know about, in-line.

Q1. If the statement encounters an error, is the _progres client AVM that is executing the statement supposed to crash altogether?  I am observing that it occasionally crashes our AVM during the COMPILE statement, and the error message in the event log seems to be pointing us to a file I/O issue.

--- No.  Of course the AVM should not crash when it encounters an error during compilation - or ever for that matter.  It should just display an error message.

Q2. ...Is it possible to compile in one operation and then use the SAVE argument in a subsequent operation to save the r-code, but avoid incurring additional CPU overhead during a subsequent COMPILE w/SAVE?  ...

--- No.  When you use the COMPILE statement, it will compile the code even if you had compiled it before and there is an in-memory version of the r-code.

Q3.a. ...When you compile a class definition file, ABL compiles the class definition file identified in the COMPILE statement and all class files in its inherited class hierarchy, by default. I'm curious about the "by default".  How do we influence whether the compiled output of multiple (inherited) classes are written or just the single class that was actually the target of the COMPILE request.  ...

--- When you compile a child class the parent(s) will also get compiled.  If you say SAVE, all levels are saved.  There is no way to make it not do this.

Q3.b. If multiple r-code files are going to be unexpectedly written to disk (without being requested) then is there an API for telling us (maybe before or after the fact) *what* particular files are being written to disk?

--- No

Q5. ... There is a MULTI-COMPILE attribute on the COMPILER handle.  First I set that to TRUE. Next I would like to compile classes A, B, and C and write them all to disk.  They are inheriting from each other like so : A->B->C.    As a preparation I first compile them all in memory without using the "SAVE" argument.  Now I want to persist the results on disk, since they are presumably stored in cached memory.  If I use "COMPILE" with the "SAVE" argument on class C, then is the expected behavior that it should only write a single r-code to disk for class C?  (Leaving A and B unwritten?)

--- I assume C is the bottom of the hierarchy.  No.  Same as the answer to Q2 and Q3.  It will compile all 3 and save all 3 to disk.

Q6.  Along the same lines as Q3.b, is there an API we can use to get the compiler to tell us (proactively) what the cls files are that it needs to work with as a dependency before it can compile the one that is ultimately requested?  I suppose this might be considered a form of "reflection", but it would be used in order to request the compilations in the correct sequence (ie. in a sequence that prevents the compiler from writing multiple output files as a result of a single COMPILE statement).

--- No.

Posted by dbeavon on 21-Oct-2019 13:18

Thanks for the feedback.   I really appreciate it.  I'm trying to fix a couple of problems with our compilation procedures.

It occurs to me that, one day, Progress may want to consider taking over the responsibility/ownership for the compilation of OE projects .  At least for larger applications it seems that the compilation process has reached a level of complexity which is beyond what OE line-of-business shops can handle on our own.  

I think the fact that Progress included PCT in the installation is a good first step, but Progress should probably start formally prescribing its use.  Progress should also offer official support for it.

Its very interesting to me that the Progress COMPILE statement (for classes) has a "unit-of-work" that writes an entire class hierarchy to disk.  ... And even beyond the classes that are being written to disk, there are lots of other classes that it has to compile in memory (eg. other class dependencies that are referenced from the same code files).  This is a big departure from the days when compilation involved only one "p-code" file.

Given the amount of work that Progress is doing for the compilation of OO classes (and the current compilation units that already include multiple class files anyway)  it seems like the next step would be to extend the compilation unit to an entire *project*.  There should be a COMPILE statement that operates on an entire OE project, rather than the individual files contained in the project.  This would remove some of the burden that OE customers face when we find ourselves in the position of trying to build our own compiler tooling.

This thread is closed