WHERE clause regression in 10.1B

Standard boolean hierarchy has OR lower priority than AND, so

x OR y AND b

is the same as

x OR (y and b)

not

(x or y) and b

so I'm a bit surprised your prior construct worked the way you say it did.

this is the biggest bug in the 4gl, as far as i can remember. it is that serious.

it is a one of the most fundamental issues that effects almost all queries and basically all programs for thousands and thousands of users.

it's just like the bi/undo-redo log not working properly, if not more.

like Tim mentioned in boolean algebra consecutive AND predicates should be treated as being in a parenthesis.

think of the where clause as a TREE of parenthesis. starting with the main parenthesis, if you will, that may also contain other sub parenthesis and so on.

if a parenthesis has both AND and OR operations, consecutive AND operations should be treated as being in their own parenthesis.

(of course, parenthesis should be resolved first).

for example:

WHERE salesorder.item = "bike" OR salesorder.duedate > 1/1/08 AND salesorder.cust = "prgs"

EQUALS

WHERE salesorder.item = "bike" OR ( salesorder.duedate > 1/1/08 AND salesorder.cust = "prgs" )

i have lots of experience in this field especially refactoring queries and where clauses.

i would love to continue this conversation and land a hand where ever i can.

could this by any chance have to do with query optimization features that you're working on ?

The problem doesn't seem to be in 10.1C.

Do we know from what version, and maybe what service pack the problem started ??

And, is it actually a bug or problem. I do understand that changes in behavior fro version ot version can be a hassle and that PSC has been uncommonly friendly in this regard, but there is a part of me that also thinks that anyone who has relied on operator precedence, instead of adding a couple of () to make it clear to everyone what the intention was, almost deserves a shot in the foot eventually.

I must be misreading something as it seemed to be working as it should.

I'd also be surprised that it didn't work in the past as well, but I usually add brackets. Apart from clarity I don't rely on my understanding of the rules an ensure it's what I want by using brackets.

Hello Thomas,

I could not disagree more.

Another way to say it is BTW we mixed up the basic math order of execution.

So that, 5 * 5 + 10 = 35 now equals 75.

Obviously, it's ridiculous.

And honestly I'm really amazed about your remark regarding code compatibility between different versions.

To put it differently it makes millions and millions of lines of code act in a whole different way.

It's a critical bug in the most fundamental, core part of all applications.

It's not a bug it's the mother of all bugs.

I am really amazed how could such a bug not be caught in QA.

And with all due respect why Simon is not sure if it's a bug or not ?

Is this already confirmed as a bug by Progress? Or an intended change in behaviour (then it would have been documented in the Release notes or similar)?

Like Miles, I have also not seen any change in behaviour and I've done a lot of query optimization work in many different Progress and OpenEdge (including 10.1B) releases and I'm pretty sure that I would have noticed a difference if there was one.

And back to Simons first post:

company=} OR company="*" AND committee = ?Board of directors?

Why don't you use a dynamic query here? Obviously you want to give the user the chance to select all companies by entering "*". If you don't filter on company, don't use it in the query.

Isn't anyone else surprised that

A. They don't realize it's even a bug. Let alone how big of a bug it is.

B. And from the mom's and pop's shop like attitude considering it's a software with millions of users.

I'd really be interested if Salvador can jump into the conversation.

Why is it so hard to believe that others did not run into this before and are interested if it's already confirmed a bug?

If it's such a big bug as you said, I would not loose any time and log it with tech support! The community is a great thing - but nobody here has the hands on OpenEdge source code and is able to fix the bug.

Mike,

My remarks were referred to Progress not you.

I also didn't imagine you had a copy of the ABL runtime source code.

And like I said earlier I have not seen them in 10.1C. But I do have several client running 10.1B.

And like you I would also like to know at what version, maybe what service pack this huge! bug got in.

And honestly I'm really amazed about your remark regarding code compatibility between different versions.

To put it differently it makes millions and millions of lines of code act in a whole different way.

My point is that in any other language than ABL which has anything like the length of history and the degree of new features, people regularly have to rewrite their applications ... it is just part of the expectation. That we don't is extraordinary.

It's a critical bug in the most fundamental, core part of all applications.

It's not a bug it's the mother of all bugs.

I know you like to get passionate about things, Alon, but I'm a little less excited. Yes, I think you should report it to Tech Support and follow through with them. Greater consistency is always better. But, it is also the case that large numbers of ABL coders have thought for years that it was best practice to always use explicit parentheses to make order of execution clear. Anyone who has followed that best practice has no issue.

So, bug, apparently, but one that is easily dealt with and which best practice makes irrelevant.

You're right

But in this case its called for.

It doesn't get more fundamental then that, queries and WHERE clauses.

And just like you shouldn't parenthesize 5th grade algebra that has both division and additions there's no reason for parenthesis in this case.

You'll find lots of WHERE clauses that aren't needlessly parenthesized written by gurus and world class experts.

And lots more in the docs, kb, PEG, progresstalk, and all other DBMS books ... and endless amount more in the real world.

Frankly I'd expect more then "Hey Tim do you think this will be a problem ?" especially from the guys writing the software.

There's alot more we have ahead of us then this. It doesn't get anymore basic then this ?

Progress is talking about a query optimizer somewhere in the future.

The amount of WHERE clause refactoring that goes into something like that is 100 times more complicated. We can't even get the basics down.

BTW has anyone actually ran into such a case ?

While I think parentheses should be used whenever there is a potential for ambiguity, I am not so religious about them in mathematical constructs because I think that operator precedence rules there are pretty broadly understood ... at least if one sticks to fairly simple combinations of either X and / with + and -. I don't think that there is broad understanding about logical operator precedence and therefore parentheses should always be used. Not because it is necessary, but because it makes it clear and unambiguous to the next person.

There should be a Prolint rule for it.

BTW, has anyone cleared up where the problem is and isn't. We seem to have a report of the issue being there in some version of 10.1B, but not in 10.1C. Is is fixed in some 10.1B service pack? I.e., is all this fuss and fury over something that has already been found and fixed?

You're right, I agree.

You should not have to know about it

but the guys writing database servers should, and alot more.

We're looking into the issue.

Simon, Please contact Tech Support and open a call with all the information to determine conclusively if there's a regression.

Thank you for bringing this to our attention. We'll keep you informed.

Regards

Salvador

Thank you all for your input

I just want to clarify that this reported WHERE clause behavior change has NOT been reproduced nor validated

The context was from a customer-partner conversation and obviously the change (if true) is a big concern to us all

This posting was primarily to gauge if 'others' had seen this behavior and I guess the conclusion is NO - and so let me go back to the customer-partner and get to the bottom of their 'real' issue - If necessary I'll work with Progress Engineering & Salvador - but ultimately I'll report back to this thread with a 'Progress approved' conclusion

Regards

Simon

It will be a bit funny if so much outrage were generated by a problem that didn't even exist...

For anyone that is interested, the latest version of Prolint has a rule which checks for mixes of AND and OR without parens. http://www.oehive.org/node/1139

A bit.

But it's a legitimate concern. If such a bug were introduced it would be a catastrophe. Sort of like if expression short circuiting were to be broken. Sure, you can code around it, and sure you could argue that it isn't documented etc, etc. But mountains of code depend on the assumption that it works in a certain way and that it will continue to work in that way. So a little hysteria at the thought that it might be broken isn't completely unreasonable.

Sure, you can code around it, and sure you could

argue that it isn't documented etc, etc. But

mountains of code depend on the assumption that it

works in a certain way and that it will continue to

work in that way.

PSC has had to back out bug-fixes because people used "buggy" behavior in their code.

Yes, but I'm not quite sure that I follow...

This isn't what I would call a '"buggy" behavior'. It is a, perhaps, undocumented behavior but, as you pointed out, there is a standard expectation about how boolean expressions should be evaluated. If that were to suddenly change it wouldn't be that prior usages were depending on a "bug".

Are we simply in vehement agreement?

Yes, but I'm not quite sure that I follow...

PSC has had to back out "bug fixes" because people came to depend on the behavior in their applications, so - in essence - I'm agreeing with a point you made in your prior post.

This isn't what I would call a '"buggy" behavior'.

It is a, perhaps, undocumented behavior but, as you

pointed out, there is a standard expectation about

how boolean expressions should be evaluated.

Every set of rules for boolean operations I've ever run into has AND as a higher priority than OR. For PSC to do it differently would be mind-boggling. That such a variance from commonly expected rules of logic could've gone un-detected for so many years even more so.

If that were to suddenly change it wouldn't be that

prior usages were depending on a "bug".

>

Are we simply in vehement agreement?

I think so.

Any conclusions to be made at this point?

I think it's fairly clear that it was a false alarm.

Thanks to all.

So far we (the originator and Progress) have not been able duplicate or reason as to why the original issue was seen/reported.

Please consider this thread as a 'false alarm' but thank you all for the community spirit

Simon

Thank you very much Simon.

Thank you for your patience and professionalism, as always.

And to everyone, sorry for all the stupid things I said.

Please don't take me seriously, no one else does.

Thank you

I don't think that Simon should get away with this so easily ....

I don't think that Simon should get away with this so

easily ....

Definitely not.

Mandatory fine of one box of dougnuts for every person who spent time responding to this thread.

A beer at Exchange and answerpoints on psdn...