R-Code and CRC Checking: A Whole New ERA for Compiled ABL Co

Posted by dbeavon on 25-Apr-2015 23:53

In OpenEdge 11.X, it became obvious to us that something dramatic changed in the way that Progress performed its so-called (and oh-so-obnoxious) "CRC" checks.

These "CRC" checks are ostensibly there to help the developer know if he did something stupid, ie. accidentally running the "wrong" code on a given OE database. But we've noticed a decreasing trend in the number of factors that can generate these "CRC" errors - meaning that Progress is getting more lenient (... not, in this case, that the developer is making fewer stupid mistakes....)

But in OE 11.X the trend changed. All of a sudden the factors causing "CRC" errors for us have practically disappeared altogether - almost like a magic. Even the "schema holes" aka "field rpos issues" hardly seem to come up anymore (you are forgiven if you don't know what I'm talking about - you can rest assured that you probably don't want to know.)

So what is up with that? Do all the tedious KB articles (about "CRC" checks and R-code compatibility rules) need to be updated to account for the new magic?

About the only thing I found in the KB was this mysterious sentence at the footnote of some extremely dated article. Notice my favorite part: *** A MISMATCHING CRC IS NO LONGER AN ERROR CONDITION ***

"In current releases (OpenEdge 11.x), due to changes to better support online schema changes and to have more portable code in DataServer environments, a mismatching CRC is no longer an error condition. Instead that triggers an extra runtime check is performed to see if the schema is compatible anyway."

In my way of reading this, the "CRC" is finally on the verge of being phased out altogether, considering it has become only one precondition to perform a more important "extra runtime check". The result of that extra check now becomes the preeminent opinion on whether the compiled code can run on a given OE database or not.

This mysterious sentence begs a lot of questions. What is occurring during that "extra runtime check"? How often does it run? Can we monitor its performance characteristics and see if/where it impacts our code? Why isn't there more information about it, especially considering the number of articles we can find related to its obnoxious "CRC" predecessor? At a minimum I would expect a dedicated KB article, if not a paragraph in the OpenEdge documentation on software deployments.

This isn't just a matter idle curiousity. If we get the so-called "CRC" errors in 11.X, it means not only that we failed the "CRC" precondition but that we failed the "extra runtime check" as well. If we don't know what is occurring in that new check, how can we remediate the runtime error in a production environment?

Can anyone provide insight into the magical "extra runtime check" that now supercedes the so-called "CRC"? It would allow me to sleep better at night knowing how it works.

All Replies

Posted by Fernando Souza on 27-Apr-2015 09:38

I am not sure if you are referring to the DataServer specific behavior, which has been relaxed in recent releases of OE 11.

However, the CRC check for the ABL against OE databases has been relaxed since OE 10.1C. There was a release note that was made available at that time but the documentation has since then been updated to include information on when the CRC mismatch may be OK. It is to allow r-code to execute when you add new fields to a table, for supporting online schema changes.

In the "Managing ABL Applications" book, under the "Appendix B: R-code Features and Functions" chapter, under the section "Database CRC changes that do not prevent r-code execution", there are two notes that explain the scenario under which the AVM will still execute the r-code:

- You add new field to a table with a _Field-rpos value greater than any other field in the table. In other words, the new field cannot be inserted into an existing _Field-rpos sequence.

- You make changes to existing fields that do not affect the values of _Field-name, _dtype, _Field-rpos, _Extent.

So as long as you maintain the field name, data type, field rpos and extent of all the fields that existed when you compiled the r-code, and any new fields added have their field rpos higher than the last field rpos value at the time of compilation, the AVM will allow that r-code to execute.

If you find knowledge base articles (or documentation) with outdated information, I suggest you contact Technical Support and bring it to their attention.

Posted by dbeavon on 27-Apr-2015 11:46

I don't think the changes I've noticed in CRC checks are DataServer-specific, although I think DataServer applications may benefit the most from the changes.

The quote I had provided is from a generic article about how/what schema info contributes to the "CRC" checks.  The KB article is P104149.

I think the documentation you pointed me to is still wrong/understated where "_field_rpos" is concerned.  It appears to me, based on experimentation, that "_field_rpos" differences are no longer a factor that will prevent r-code execution.  Ie. based on the subtle footnote in the KB article, Progress runs an "extra runtime check" and determines that schema is compatible anyway, despite any "_field_rpos" differences.

So it does NOT appear necessary to maintain the field rpos that existed when you compiled the r-code. The AVM will allow that r-code to execute anyway. This became evident when we noticed that we could use a DF schema dump of a production database to compile source code against (ie. after creating a schema-only database).  OE now allows us to release that compiled  r-code to run on the actual production database.  Previously this was prevented because of very obscure issues in "_field_rpos"

Posted by Fernando Souza on 27-Apr-2015 12:13

That is not true. We still rely on the field rpos and will/should not allow the r-code to run if the field rpos values do not match. We need to validate the field rpos otherwise that could lead to corruption of data, where the data could end up in the wrong field.

When you dump the .df of a table, the .df carries the field rpos value for the fields with it (unless you chose not to keep the POSITION values) so it is expected that you load the .df into a db, compile the r-code and are able to run the same r-code.

It would be interesting to see exactly the situation you are referring to, to see if it doesn't fall into the case described in the documentation or it is something else. if you find a case where the field rpos on an existing field actually changed and you were allowed to run the existing r-code, then you should log a call with Technical Support because that is not supposed to happen.

Posted by David Moloney on 27-Apr-2015 21:46

With regards to DataServers ... it's CRC check was always more rigorous than applications compiled against an OE database.  This began to get dismantled in OE 11.2 when we added "position-independent'" r-code, meaning that the physical position of a field in the server database could change (as long as you re-execute a schema pull) to realign _Field-rpos with the new physical position on the server.  

Starting in OE 11.5, DataServers have removed all the old restrictions on CRC such that the CRC requirements are now the same as OpenEdge.  You can make any changes to existing fields that you want as long as you don't affect the values of _Field-name, _dtype, _Field-rpos, _Extent, just like OE.  You can also add fields without triggering a CRC mismatch or you can delete a field as long as there are not references to it in your r-code.  

The DataServer documentation describes these features in a section called, "Compiling OpenEdge procedures".

Posted by dbeavon on 28-Apr-2015 12:33

<p>Again, I don't think we are talking about DataServers only.  My repro is for an OE 11.3 database and OE 11.3 ABL.</p>

<p>Apparently I'm not the only one with confusion.  What can we do to request more documentation on the changes from Progress?  We actually have a software deployment model that relies on the changes in behavior and it is extremely worrisome that it is totally undocumented.</p>

<p>Here is a repro that used to generate CRC errors and does not anymore.</p>

<p> /* Prog: */

DEFINE VARIABLE v_Number AS CHARACTER NO-UNDO.

MESSAGE "Enter a Number: " UPDATE v_Number FORMAT "X(10)".

CREATE TestTable.

TestTable.TestField1 = v_Number.

TestTable.TestField3 = INTEGER(v_Number).

</p>

<p> /* DB #2 */

CREATE TABLE PUB."TestTable"(

"TestField1" VARCHAR (20) DEFAULT '' PRO_ORDER 10 PRO_CASE_SENSITIVE 'N' PRO_FORMAT 'x(8)'  )

AREA "ddn_log" PRO_HIDDEN 'N' PRO_LABEL 'TestTable' PRO_DUMP_NAME 'TestTable'

ALTER TABLE PUB."TestTable" ADD COLUMN ( "TestField2" BIT DEFAULT 'no' PRO_FORMAT 'yes/no'  )

ALTER TABLE PUB."TestTable" ADD COLUMN ( "TestField3" INTEGER DEFAULT '0' PRO_FORMAT '->>,>>9'  )

ALTER TABLE PUB."TestTable" DROP COLUMN  "TestField2"

</P>

<p> /* DB #1 */

CREATE TABLE PUB."TestTable"(

"TestField1" VARCHAR (20) DEFAULT '' PRO_ORDER 10 PRO_CASE_SENSITIVE 'N' PRO_FORMAT 'x(8)'  )

AREA "ddn_log" PRO_HIDDEN 'N' PRO_LABEL 'TestTable' PRO_DUMP_NAME 'TestTable'

ALTER TABLE PUB."TestTable" ADD COLUMN ( "TestField3" INTEGER DEFAULT '0' PRO_FORMAT '->>,>>9'  )

</P>

<P> /* Different R-pos : */

for each _field no-lock where _field._field-name begins "Test":

display _field._field-name _field._field-rpos _field._file.

end.

┌──────────────────────────────────────────────────────┐

│Field-Name                       field-rpos File-recid│

│──────────────────────────────── ────────── ──────────│

│TestField1                                2      15209│

│TestField3                                4      15209│

┌──────────────────────────────────────────────────────┐

│Field-Name                       field-rpos File-recid│

│──────────────────────────────── ────────── ──────────│

│TestField1                                2      15209│

│TestField3                                3      15209│

</P>

<

Posted by dbeavon on 28-Apr-2015 13:00

Just to review, the "TestTable" is created via simple SQL deltas.  In the case of DB2, an intermediate TestField2 is dropped.  In the case of DB1 no TestField2 was ever created in the first place.

DB2 leaves an rpos schema "hole", as they used to be called (based upon what little I know about such things).

Compiled r-code against one can be executed on the other and vice-versa without CRC errors. This is exactly what the KB article on the topic had said (P104149).  Can someone please independently verify this repro, since Fernando Souza and I aren't on the same page yet?  Can someone also please read the KB article (P104149) - which is predominantly NOT about DataServers?  Please let me know if you read the footnote in the same way that I do - ie. that CRC's alone will not prevent code execution, but will trigger an "extra runtime check".

My biggest concern is relying on a behavior whose only documentation is (at this point) just a subtle footnote in a KB article.

A secondary concern about this ABL change is that, while it is a very welcome one, it is not without its issues.  While CRC checks are not preventing code execution, they are allowing yet another problem to occur.  If you run one of the rcodes you get an error (the one compiled with larger field rpos positions).  This won't run on the DB #1 with the smaller field rpos.

I get the following:

SYSTEM ERROR: bffld: nxtfld: scan past last field. (16)

This seems like an error you'd rather not encounter if it can be identified earlier.

Posted by Libor Laubacher on 28-Apr-2015 13:16

Ø  Can someone please independently verify this repro

 
Yes and yes on the bffld err.
 
[collapse]
From: dbeavon [mailto:bounce-dbeavon@community.progress.com]
Sent: Tuesday, April 28, 2015 8:02 PM
To: TU.OE.Development@community.progress.com
Subject: RE: [Technical Users - OE Development] R-Code and CRC Checking: A Whole New ERA for Compiled ABL Code!
 
Reply by dbeavon

Just to review, the "TestTable" is created via simple SQL deltas.  In the case of DB2, an intermediate TestField2 is dropped.  In the case of DB1 no TestField2 was ever created in the first place.

DB2 leaves an rpos schema "hole", as they used to be called (based upon what little I know about such things).

Compiled r-code against one can be executed on the other and vice-versa without CRC errors. This is exactly what the KB article on the topic had said (P104149).  Can someone please independently verify this repro, since Fernando Souza and I aren't on the same page yet?  Can someone also please read the KB article (P104149) - which is predominantly NOT about DataServers?  Please let me know if you read the footnote in the same way that I do - ie. that CRC's alone will not prevent code execution, but will trigger an "extra runtime check".

My biggest concern is relying on a behavior whose only documentation is (at this point) just a subtle footnote in a KB article.

A secondary concern about this ABL change is that, while it is a very welcome one, it is not without its issues.  While CRC checks are not preventing code execution, they are allowing yet another problem to occur.  If you run one of the rcodes you get an error (the one compiled with larger field rpos positions).  This won't run on the DB #1 with the smaller field rpos.

I get the following:

SYSTEM ERROR: bffld: nxtfld: scan past last field. (16)

This seems like an error you'd rather not encounter if it can be identified earlier.

Stop receiving emails on this subject.

Flag this post as spam/abuse.

[/collapse]
Posted by dbeavon on 29-Apr-2015 07:26

So what is the next step?  Am I correct to say that this confusion needs to be addressed with either a KB or new documentation?  I would have opened a support case by now but somehow Progress decided to disable my SupportLink based on my customer serial number or something.  Very frustrating.

Posted by Libor Laubacher on 29-Apr-2015 07:48
You have been told the reason 3 months ago. I can resend that email to you to refresh your memory, but in a nutshell – the company your work for is and end-user/customer of another company. End-users (unless they are direct end-users) do not have SupportLink account/can’t open TS cases and have to open their cases thru the company they purchased the license/software from.
 
So if you wish the open TS ticket (as you should), do it thru them.
 
[collapse]
From: dbeavon [mailto:bounce-dbeavon@community.progress.com]
Sent: Wednesday, April 29, 2015 2:27 PM
To: TU.OE.Development@community.progress.com
Subject: RE: [Technical Users - OE Development] R-Code and CRC Checking: A Whole New ERA for Compiled ABL Code!
 
Reply by dbeavon

So what is the next step?  Am I correct to say that this confusion needs to be addressed with either a KB or new documentation?  I would have opened a support case by now but somehow Progress decided to disable my SupportLink based on my customer serial number or something.  Very frustrating.

Stop receiving emails on this subject.

Flag this post as spam/abuse.

[/collapse]
Posted by dbeavon on 29-Apr-2015 07:58

Yes please resend.  

I'm told by the people here that keep track of it that we have a written contract that says we have access to Progress support.  

Nobody who has to do development work with OpenEdge products should go without support.  There are too many, shall we say, "issues" that come up.  How is it that you sell OE development licenses to your so called "end users" and then tell them they aren't supported?

Posted by Libor Laubacher on 29-Apr-2015 08:03
>> 
Nobody who has to do development work with OpenEdge products should go without support.  There are too many, shall we say, "issues" that come up.  How is it that you sell OE development licenses to your so called "end users" and then tell them they aren't supported?
<< 
 
If you wish to discuss this further, or believe this is incorrect or you feel that you have been mis-treated, please contact your account manager to get this sorted out/clarified. This won’t get solved here.
 
[collapse]
From: dbeavon [mailto:bounce-dbeavon@community.progress.com]
Sent: Wednesday, April 29, 2015 2:59 PM
To: TU.OE.Development@community.progress.com
Subject: RE: [Technical Users - OE Development] R-Code and CRC Checking: A Whole New ERA for Compiled ABL Code!
 
Reply by dbeavon

Yes please resend.  

I'm told by the people here that keep track of it that we have a written contract that says we have access to Progress support.  

Nobody who has to do development work with OpenEdge products should go without support.  There are too many, shall we say, "issues" that come up.  How is it that you sell OE development licenses to your so called "end users" and then tell them they aren't supported?

Stop receiving emails on this subject.

Flag this post as spam/abuse.

[/collapse]
Posted by Roger Blanchard on 29-Apr-2015 08:09

"There are too many, shall we say, "issues" that come up".

There are no "issues" in any other development environment....

Posted by dbeavon on 29-Apr-2015 08:14

I'm looking for constructive ways of moving forward with scary and misleading error messages that seem to say we are experiencing database corruption (although that is not the case).

Would someone with access to Progress Support please see what we can do to get the ball rolling?  To start with it would be nice to see a KB or documenation to expand on the surprising statement that "A MISMATCHING CRC IS NO LONGER AN ERROR CONDITION ".  If we move forward then some day we will have one less issue on our hands.

Posted by Libor Laubacher on 29-Apr-2015 08:59
I have logged PSC00331998 for the SYSTEM ERROR: bffld: nxtfld: scan past last field. (16)
 
[collapse]
From: dbeavon [mailto:bounce-dbeavon@community.progress.com]
Sent: Wednesday, April 29, 2015 3:15 PM
To: TU.OE.Development@community.progress.com
Subject: RE: [Technical Users - OE Development] R-Code and CRC Checking: A Whole New ERA for Compiled ABL Code!
 
Reply by dbeavon

I'm looking for constructive ways of moving forward with scary and misleading error messages that seem to say we are experiencing database corruption (although that is not the case).

Would someone with access to Progress Support please see what we can do to get the ball rolling?  To start with it would be nice to see a KB or documenation to expand on the surprising statement that "A MISMATCHING CRC IS NO LONGER AN ERROR CONDITION ".  If we move forward then some day we will have one less issue on our hands.

Stop receiving emails on this subject.

Flag this post as spam/abuse.

[/collapse]
Posted by Fernando Souza on 05-May-2015 16:01

First, thanks for the reproducible.

Just to close the loop on this, the reproducible provided shows a bug. As I explained previously, this is *not* supposed to happen. The caveat is that the bug only happens when you create the table via SQL. If you create the same table via an ABL client, you will get the expected CRC error. Therefore, once we fix the bug that is allowing this r-code to execute, you will *not* be able to run it, and that is what is expected behavior. The error you reported (16) is one of the many issues that can happen if we do not stop you from running the outdated r-code. The documentation I pointed to (that describes the cases where we allow you to still run the r-code (i.e. new fields added)) is still accurate. You just ran into a bug (which has already been logged) that needs to be addressed.

Posted by dbeavon on 06-May-2015 00:24

Thanks Fernando.  I am optimistic for a resolution to that open bug.

I have an open case related to the larger issue - finding out if/when CRC's are exclusively used to disqualify programs from being executed.  I think this is still a topic of confusion.  I'll definitely follow up and let you know what comes out of that case.  According to that KB article, (000010488, What Metaschema fields are involved in table and database CRC calculations) it seems that CRC's will now play a subordinate role in determining if r-code is compatible.  CRC's no longer create an error condition or, by themselves, prevent code from running on a given database.  The "extra runtime check" is apparently what really matters (going forward).

From what I can tell, it looks like we've reached a tipping point in the old "CRC"-based strategy.  Progress has continually relaxed previous constraints which had inhibited online dictionary changes, notably those kinds of changes made dynamically via SQL DML and those not requiring exclusive schema locks.

The expectation I have is that I should be able to (1) export my schema definitions from a source database to a schema-only database, (2) compile against it and then, (3) the result should be r-code that is compatible with the source.  This is not something that was possible until the very most recent releases of OE.  My recollection is that the scenario was prevented because of some of the more unusual OE schema internals (OE _field-rpos holes etc - things which aren't exposed very well, and whose values can't be influenced easily by a developer).

Posted by dbeavon on 03-Jun-2015 07:44

There is now a KB on the bug that was reported:

Running r-code leads to SYSTEM ERROR: bffld: nxtfld: scan past last field. (16) http://knowledgebase.progress.com/articles/Article/Running-r-code-leads-to-SYSTEM-ERROR-bffld-nxtfld-scan-past-last-field-16

Also the following two articles were updated with the information gathered while working on this issue. What I gather now is that the logical field positions (_field-rpos aka POSITION) are as important as they always were (and as important as the datatype) for the purpose of detecting incompatible .r-code. Any differences in these field positions will generate a CRC mismatch AND prevent .r-code from running. There are other schema-related values that may change, generate CRC mismatch, but still allow old .r-code to remain compatible and continue to be executed. The _field-rpos does not fall into this category in OE 11.3 anymore than it did in OE 11.1.

What Metaschema fields are involved in table and database CRC calculations? http://knowledgebase.progress.com/articles/Article/P104149 Cyclic Redundancy Check Explained http://knowledgebase.progress.com/articles/Article/20245
This thread is closed