ABLUnit - Regression or not ?

Posted by Riverside Software on 07-Apr-2020 11:57

Up to 12.1, failure to compile a unit test resulted in a job failure:

test1:
[ABLUnit] Could not locate element 'isTue' in class 'OpenEdge.Core.Assert'. (12927)
[ABLUnit] QUIT statement found

BUILD FAILED

Since 12.2, this is not the case anymore, there's just a line in the log output:

test1:
[ABLUnit] Unable to build type info for C:\Users\gquer\projets\pct\testbox\ABLUnit\test8\src\ABLUnitTest.cls: Could not locate element 'isTue' in class 'OpenEdge.Core.Assert'. (12927)
[ABLUnit] QUIT statement found
[ABLUnit] Total tests run: 0, Failures: 0, Errors: 0

BUILD SUCCESSFUL

I'd consider that a potential problem, as I suspect that almost nobody read the output of the unit tests (that's usually buried in a very long log file). Was that done on purpose ? Or is it an unintended side effect ?

OpenEdge Development - Forum

All Replies

Posted by Peter Judge on 07-Apr-2020 13:24

Deliberate change, based on the fact that the error handling in ABLUnit wasn't very robust, and stuff was getting swallowed/discarded/lost.
 
The outcome should still be the same: if "test1" fails to compile, then the whole job should fail. That message just had some info added to figure out what had failed (and why); it didn't change the nature of the failure.
 
 
There's a relnote  for issues  OCTA-19255, OCTA-19708  in docs.progress.com/.../PROGRESS_OE_12.2_RELEASE_NOTES.pdf , which links to docs.progress.com/.../Support-for-multiple-lifecycle-methods-in-ABLUnit.html (that last link is broke but I've logged an issue to fix).
 
 
 
 
 

Posted by Lars Neumeier on 07-Apr-2020 13:58

docs.progress.com/.../Updates-to-ABLUnit.html

Posted by Riverside Software on 07-Apr-2020 13:58

Then there's definitely a regression. My test case is here: github.com/.../test8

I expect a build failure to happen, as the .cls file which is tested doesn't compile. Test is failing on 12.2, and only on this version. I've tried the Progress version of ABLunit task, same problem, there's no failure:

test1:

 [ablunit] Running C:\Users\gquer\projets\pct\testbox\ABLUnit\test8\src\ABLUnitTest.cls

BUILD SUCCESSFUL

Posted by Peter Judge on 07-Apr-2020 14:10

Ah, cool. Thanks Lars.
 

Posted by Peter Judge on 07-Apr-2020 14:58

A difference between the versions is that there's always a results.xml now written, even if the tests fail to compile. I'm guessing that the 'success' checks check for the existence of that file … which it seems to in ABLUnit.java, where there's no BuildException thrown.
 
 
 
This test should pass today, maybe that's what's happening?
Assert:isTue(TRUE).
 
 
 
(kidding!)
 
 
 
 

Posted by Riverside Software on 07-Apr-2020 15:20

Results.xml is generated, but no mention of the error in the file.

Posted by Riverside Software on 07-Apr-2020 15:20

Results.xml is generated, but no mention of the error in the file.

This thread is closed