How to find CLASS definition file .cls to be compiled before

> I am looking for an option in COMPILE statement that for a given .p will provide me the list of .cls, CLASS definition files, that I need to recompile first in order to do a successful .p compile.  Can I get that with my current 11.7...?

Not as far as I'm aware (though I would be happy to hear that I'm wrong).

That would be great to have natively.  In general terms I'd call that "impact assessment".  Would be very useful for includes as well.  E.g.: given some propath and the modules in it, if I were to change foo.i, which compile units would require recompilation?

E.g.: given some propath and the modules in it, if I were to change foo.i, which compile units would require recompilation?

This category of question is, of course, one of the primary motivations of the aforementioned tools. :)

Depending on the size of your application, the simplest answer may be to compile everything. This is the direction that CI/CD pushes. Make a change to anything, trigger a compile all on a server, run automated testing, review results, then fix issues and run again or deploy the code.

If your application takes more than a few minutes to compile, you'd need to have that ability to incrementally build it based on only what changed like you're asking about. However, you probably should still have a dedicated step in your development process that recompiles the entire application before it reaches the production environment. And if you have that step in your process, why not just use it to catch this kind of change?

The change to include file (foo.i) triggers whole application recompile. With 10,000+ programs that is between 20 minutes and 1.5 hours, depending with local database or with remote db. We used to have XREF system that tells which programs are affected by include file change. We scraped it more than 10 years ago and do a whole system compile for includes since. 

When we deploy to production 1 program.p we used to compile that one program alone. Now we could not do that.

"What I am looking for is not a third-party tool (RoundTable..."

For any interested parties following along, this is the type of functionality that Roundtable provides.

You might be able to get that from compile with the xref-xml option.

docs.progress.com/.../COMPILE-statement.html

I also wrote an xref2tt that takes non-xml xref output and converts it to temp tables. This predates classes and OO stuff so it may need updating.

community.progress.com/.../1614.xref-parser

[quote user="Dmitri Levin"]

I am looking for an option in COMPILE statement that for a given .p will provide me the list of .cls, CLASS definition files, that I need to recompile first in order to do a successful .p compile. Can I get that with my current 11.7 or future versions of Progress?

[/quote]

I am trying to understand your requirement better. COMPILE statement looks at a single file with in the context of the PROPATH. Since ABL is both interpreted and compiled, you don't have to compile all the referenced classes to compile a .p. Does this not work for your requirement?

Are you compiling part of your application/project for separate packaging purposes?

For the reverse use case, given a class file is changed what files need to be recompiled, it can't be language level feature as the COMPILE statement works on one file at a time and doesn't have the concept of project. This needs to be handled by tooling. We are planning to add this to PDSOE and Gradle tasks in future.

Thanks,
Ravi

"Given a class file is changed what files need to be recompiled, it can't be language level feature as the COMPILE statement works on one file at a time and doesn't have the concept of project. This needs to be handled by tooling. We are planning to add this to PDSOE and Gradle tasks in future."

Roundtable does a great job at this.

>> I am trying to understand your requirement better. COMPILE statement looks at a single file with in the context of the PROPATH. Since ABL is both interpreted and compiled, you don't have to compile all the referenced classes to compile a .p. Does this not work for your requirement?

I agree that the requirements aren't 100% clear.  It sounds like a question related to *production* deployments ... but the technique/strategy of partial compilation sounds like something one would only be attempting in a *development* environment.  For *production* purposes, you shouldn't be taking risks, and you should compile "everything" (whatever that may mean in your case).  Start things out by clearing the r-code in the PROPATH so you don't have any cruft remaining from a prior build.

Does your solution involve one monolithic project with tens of thousands of programs?  It might be more manageable to break things up into smaller projects that are managed as deployment units.  If you need to deploy a change to a project that is at the top of the dependency stack (for example in U/I -related code only) then you can use your discretion to limit the amount of compilation that is done.  Just deploy one of the many projects.  

A *single* ABL program should NEVER be a deployment unit.  That may have worked twenty years ago, but it is a losing strategy these days with so much code re-use and multiple layers of software that depend on each other.

>> Depending on the size of your application, the simplest answer may be to compile everything. This is the direction that CI/CD pushes

Yes, compile "everything".  Compilation is CPU-bound so get a build server machine with plenty of cpu's.  Every time you double the CPU cores, you cut the compile time in half.  Pushed to the extreme you may run into a Progress bug related to concurrent use of actively-changing r-code files but there is probably a workaround for that if you encounter it (see community.progress.com/.../59721 ).

>> With 10,000+ programs that is between 20 minutes and 1.5 hours, depending with local database or with remote db.

IMHO you should always compile with *local* schema/database.  

Export the database schema, wherever it is, and recreate the empty version of the database locally to compile against.  Another trick is you don't even need to proserve the database to compile code against it from multiple concurrent clients.  Just use "-1" along with "-RO".  This allows every ABL client that is compiling code to access the database in "single-user" mode while performing the COMPILE operations.  

The re-creation of the local database is CPU-bound on a single core, and can be a small bottleneck.  But it will definitely save you some time in the long run if you have tens of thousands of programs to compile.

Most of this is not terribly hard to do, but I can see why there are third-party tools that are filling the gap.  It is a really big shame that Progress isn't stepping up to the plate and offering their own toolchain for compiling ABL.  The only recent change that I'm aware of is that they now redistribute PCT in their installation packages.  But they don't formally support it in tech support, last time I asked.

In general, I agree that a *single* ABL program should never be deployed alone. However, what about that bug that makes it through all the builds/tests and is only discovered by an end user? You can't prevent all bugs. Suppose this bug is high-impact. We'll need to point fingers later, but right now we need to fix the bug and deploy immediately. How does one deploy a fix for that high-impact bug immediately to stop the leak? The fix should ultimately make its way through the regular build process that compiles everything and runs all the tests, but sometimes you know what's wrong, fix it, and need to deploy it now - not in 20 minutes or 2+ hours. What's more important - following the letter of the law on CI/CD process, or getting your application back online? I feel like this type of scenario is what Dmitri is encountering.

Rereading your post, @dbeavon, it sounds like breaking up the code base is part of the answer, but that could certainly be a challenge to do as well (and maybe more work than figuring out the file dependencies).

Tim

"For classes and procedures, it could be any variables (and properties) defined as OOABL types, and any return values or parameters defined (that are OOABL types).  Tools (anyone's) could consume that output (find any r-code and check the last-compiled date?) and do the necessary work - whether that's building a single program and its dependencies, or building a tree-of dependencies."

Tools that do that are really cool.

Peter, note that for class dependencies one wants code on which the current one depends, but for include files it is the reverse, i.e., all the code which depends on the current one.  I don't think that is feasible without a database.

It looks like I started a big conversation in the area where I absolutely know nothing.

Before we started CLASSES, if a program does not recompile then something is wrong in it or it's includes.

Now with CLASSES, if a program does not recompile then the problem could be in CLASS definition and who knows were it is.

While include file(s) are written in the program code, class definition is not. That is where I am coming from.

When we comile all code, we first compile all .cls files and then all .p files.

BTW is .cls a stadart Progress extention for class definition files or that is our programmers invention?

>Posted by Ravi Sankar

>you don't have to compile all the referenced classes to compile a .p. Does this not work for your requirement?

It works fine, except when programmers change class definition (.cls) and program and only tell me the program to compile :)

Since we implemented stratagy "Compile .cls first, .p second" thing become much better. Thanks.

>Are you compiling part of your application/project for separate packaging purposes?

Yes

>Posted bydbeavon

>A *single* ABL program should NEVER be a deployment unit.

Unfortunately I could not agree for the same reasons that Tim Hutchens wrote above.

Every project we deploy go with immediate bug fix and then a couple more fixes that need to be done quickly.

We do deploy "a program" all the time.

>fix it, and need to deploy it now

That is us !

>Posted by dbeavon

> IMHO you should always compile with *local* schema/database.

Interesting idea. Why I did not think about it before? Thanks.

>Posted byPeter Judge

>Are you thinking of something like a ( completely pseudo-code)

>COMPILE foo.p LIST-DEPENDENCIES

Yes, I guess that would be nice. If we could not find a class definition ( .cls file) then at least we can get the class name.

>Peter, I’d rather have that clearly in the XML-XREF than in yet another output file of the COMPILE statement.

I would go with what ever is available. Can I get it (class reference) in XML-XREF now, I that needs an enhancement request?

And thanks, I guess you all answered my question mostly.

It looks like I started a big conversation in the area where I absolutely know nothing.

Before we started CLASSES, if a program does not recompile then something is wrong in it or it's includes.

Now with CLASSES, if a program does not recompile then the problem could be in CLASS definition and who knows were it is.

While include file(s) are written in the program code, class definition is not. That is where I am coming from.

When we comile all code, we first compile all .cls files and then all .p files.

BTW is .cls a stadart Progress extention for class definition files or that is our programmers invention?

>Posted by Ravi Sankar

>you don't have to compile all the referenced classes to compile a .p. Does this not work for your requirement?

It works fine, except when programmers change class definition (.cls) and program and only tell me the program to compile :)

Since we implemented stratagy "Compile .cls first, .p second" thing become much better. Thanks.

>Are you compiling part of your application/project for separate packaging purposes?

Yes

>Posted bydbeavon

>A *single* ABL program should NEVER be a deployment unit.

Unfortunately I could not agree for the same reasons that Tim Hutchens wrote above.

Every project we deploy go with immediate bug fix and then a couple more fixes that need to be done quickly.

We do deploy "a program" all the time.

>fix it, and need to deploy it now

That is us !

>Posted by dbeavon

> IMHO you should always compile with *local* schema/database.

Interesting idea. Why I did not think about it before? :) Thanks.

>Posted byPeter Judge

>Are you thinking of something like a ( completely pseudo-code)

>COMPILE foo.p LIST-DEPENDENCIES

Yes, I guess that would be nice. If we could not find a class definition ( .cls file) then at least we can get the class name.

>Peter, I’d rather have that clearly in the XML-XREF than in yet another output file of the COMPILE statement.

I would go with what ever is available. Can I get it (class reference) in XML-XREF now, or that needs an enhancement request?