Data validation approach

Posted by Peter Judge on 11-Dec-2015 09:54

I've got a bug to fix that requires me to add some validation into a property setter. The challenge is that the object is a general-purpose value object (it's an HttpHeader) and so the property (the Value) can have different validation depending on the Name of the header.

I can see a couple of approaches to providing validation for particular headers

0) Inline validation. aka the ugly way. Add a CASE statement in the header object or try to patch it wherever the value might be set.   Actually no ... not gonna do that :)

1) Subclass the HttpHeader object and add an internal validation method . This isthe easiest and is well-encapsulated. The challenge is that  a developer/consumer of these header objects has to know to create a ContentTypeHeader instead of an HttpHeader. That said, I have infrastructure already in place to deal with this (an HttpHeaderBuilder) but there's still no guarantee that it will be used.

Which leads me to  ...

2) Provide a independent Validation module, that takes a value and returns something if the validation fails. Similar to assertions but with the capability to have more specialised logic.

This validation logic can be called from various places (before setting the value, before creating output etc) as a double-check that values are correct.

This also gives me a way to have validation for a particular thing in ONE place (as opposed to checking it everywhere its set).

This requires creating various validation objects which would (in this particular case) be stored and keyed off of the header name (Content-Type is the one I care about right now).

I would also modify the HttpHeader to call this validation logic.

3) Something completely different.

Any thoughts/comments? 

All Replies

Posted by Tim Kuehn on 11-Dec-2015 10:49

My usual practice is to separate the domain and validation logic so I can use either of them when and how I wish.

The advantage is that I can populate the domain instance in an incremental manner that could include contextual circumstances that would otherwise be illegal -or- I could use the same domain object different ways and then validate it according to the validation rules that apply in the current context by using the appropriate validator class.

This structure not only results in more flexibility, it also allows me to use the validation code to check other non-domain objects & contexts.

Splitting the two types of logic allows you to go with both of your proposals - subclass the current class, create the validation logic as a separate class, and then either have the new ContentTypeHeader subclass call your validation logic

-or-

the customer developer can pass their HttpHeader class to your new ValidateHttpHeader class to do the check at the appropriate time.

Posted by Peter Judge on 11-Dec-2015 11:09

I think I arrived at the same place: which is to create independent/external validation logic that can be called from anywhere.
 

Posted by agent_008_nl on 12-Dec-2015 00:09

Maybe the start of a small rules engine written in abl (maybe you remember I made one)? You hardcoded the rule I suppose?

Posted by Peter Judge on 14-Dec-2015 08:18

I'd planned to have small, individual pieces of code logic in individual validation procedures/methods. So one would be 'is not empty" (for isntance), and another would be "has a valid mime type".

It's an interesting question as to where the line between this and a rules engine is.

Posted by agent_008_nl on 14-Dec-2015 09:17

Maybe the line is: make the rules maintainable for customers / others?  So externalize it from the applicationcode and put some  dynamic evaluation for the rule in your code. The rule (can be put in the db) could be something like "customer.credit > 1000 and customer.credit < 2000". I have rules like this in the db and copy them to a dataset at as startup for quick access by the engine. And something like a flag rule active.

Posted by Peter Judge on 14-Dec-2015 09:37

The approach I'm thinking about is something like this:
 
A number of Validation classes that each perform some small validation, and potentially return one or more ValidationErrors. Each Validation takes an argument against which to perform the validation code.
A Validator type that contains a set of Validation instances and loops through them all, validating and gathering errors as it goes. The Validator type would be where you can add custom/extra validation rules.
 
When a (say) save operation is performed, get an object's Validator and do the validations within. The fun part now is  figuring out how to generically associate Validators with other objects.
 
 
 

Posted by Tim Kuehn on 14-Dec-2015 10:01

[quote user="Peter Judge"] I'd planned to have small, individual pieces of code logic in individual validation procedures/methods. So one would be 'is not empty" (for isntance), and another would be "has a valid mime type". [/quote]

I'd surmise Empty / Filled would be an object state while "valid mime type" would be a validation as the domain object might be used for multiple different types of content.

Posted by Tim Kuehn on 14-Dec-2015 10:38

[quote user="Peter Judge"] A Validator type that contains a set of Validation instances and loops through them all, validating and gathering errors as it goes. The Validator type would be where you can add custom/extra validation rules.

 When a (say) save operation is performed, get an object's Validator and do the validations within. The fun part now is  figuring out how to generically associate Validators with other objects. [/quote]
I'm wondering if the Decorator pattern would be appropriate here as it would allow you to create arbitrary stacks of functionality / validations at run time. 

Posted by Peter Judge on 14-Dec-2015 11:19

Possibly.
 
There are some challenges with being able to decorate property setters since properties cannot really be overridden unless they're abstract and even then, only once.
 
I really like the simplicity of properties, but the OOABL tugs me towards the Java approach (private variable + Get and Set methods). Blech.
 
 
 

Posted by agent_008_nl on 15-Dec-2015 00:51

> I'd planned to have small, individual pieces of code logic in individual validation procedures/methods. So one would

> be 'is not empty" (for isntance), and another would be "has a valid mime type".

 

Maybe I should work out my suggestion a bit? Your assertion could look like this:

lOk = phBuffer:find-unique('where rowid(' + phBuffer:name + ') = to-rowid(' + quoter(cRowid)  + ') and ' + cRule) no-error.

cRowid could be replaced with string(phbuffer:rowid)

You can test both "is not empty" and "has a valid mime type" with this code + rules like

cRule = "lookup('text/html,video/animaflex etc etc',something.mimetype) > 0"  

"lookup(something.mimetype, 'text/html,video/animaflex etc etc') > 0"

cRule = "something.notemptyfield <> '' "

I added the small ppt of my presentation in Brussels 2013 (emea pug) to make it a bit more clear, see the last two pages.

There will be validations that you cannot do with this code. For these you need other pieces of validation logic that you can call dynamically based on what's in your rule. Your validation module can take the dataset to validate as input. It should query the dataset with rules to validate fieldlevel, recordlevel and eventually datasetlevel (cardinalities f.e.: an order has to have at least one orderline associated).

A temptable with rules could look like the one here: https://community.progress.com/community_groups/mobile/f/17/t/16432

Of course you could hard-code the creation of a couple of rule-records for the time being, and remove them if ever you / others create a brms (are you working on aetf mobile?).

Mimetype could be an array column in the database / dataset if the beloved db would support variable size arrays. (there he goes again  ;-)   Postgresql does:

http://www.postgresql.org/docs/9.1/static/arrays.html . :-)

Imagine a new mimetype being necessary. Wouldn't it be nice if you only had to insert it into the db to get your validation ready? And wouldn't it be nice if your validationrules could be used on a non-abl client? F.e. in the HTML5 Constraint Validation API? I use them for that.

  Eagerly leaving the thinking about OO implementations to Tim and you. ;-)

btw small test proc:

/*---    File        : ruletest.p--------*/

define temp-table customer no-undo

  field mimetype    as character

  field reliability as character.

 

define variable rule    as character no-undo.

define variable lok     as logical   no-undo.

define variable hbuffer as handle no-undo.

create customer.

assign

  customer.mimetype    = 'text/html'

  customer.reliability = 'x'

  .

rule = "lookup(customer.mimetype, 'text/html,video/animaflex') > 0 and customer.reliability <> ''". /* rules like this reside  in the dataset */

hBuffer = buffer customer:handle.

lOk = hBuffer:find-unique('where rowid(' + hBuffer:name + ') = to-rowid(' + quoter(string(hbuffer:rowid))  + ') and ' + Rule) no-error.

display lok.

 
--
Kind regards,
 
Stefan Houtzager
 
Houtzager ICT consultancy & development
 
www.linkedin.com/in/stefanhoutzager

[View:/cfs-file/__key/communityserver-discussions-components-files/19/MDD-and-BRM-in-a-new-OE-backend.pptx:320:240]

Posted by gus on 15-Dec-2015 08:59

> On Dec 15, 2015, at 1:51 AM, agent_008_nl wrote:

>

> Imagine a new mimetype being necessary. Wouldn't it be nice if you only had to insert it into the db to get your validation ready?

to accomplish that, all you need is a table with a single character colum. insert a row for each valid mime type you want to be able to handle. then to validate, all you need is a can-find (mimetype).

Posted by Peter Judge on 15-Dec-2015 09:34

Alas, one doesn't always have a database (or rather, one with your data in it).
 

Posted by gus on 15-Dec-2015 09:59

> On Dec 15, 2015, at 10:35 AM, Peter Judge wrote:

>

> Alas, one doesn't always have a database (or rather, one with your data in it).

of course you do. everyone must have a database. don't they ???

anyhow, agent_008 posited that one existed.

Posted by agent_008_nl on 15-Dec-2015 10:00

The data can be read from xml / json also, of course.

Posted by agent_008_nl on 15-Dec-2015 10:02

+ you could forget about the db for the while being and create the temptables, uh, the records, hardcoded.

Posted by agent_008_nl on 15-Dec-2015 10:30

I don't know what you mean. I have these data in a separate db. Moreover they could be in json/xml instead of a db. The data used for validations reside in a dataset, in memory. I'm using a version number also, to be able to notify updates in the db.

Posted by Marian Edu on 15-Dec-2015 12:20

I might be missing the point of this post all together, maybe the title is misleading or something but...

use external validators to enforce valid object state, use anything that looks like a rule engine to do that, private variables with getter/setter bleah???

Come on Peter, should I send some palinca over to help you through boring mondays? :)

Beside there is nothing wrong with using non-standard content types so what are you trying to enforce there?

www.w3.org/.../4_Content-Type.html  

Posted by Peter Judge on 15-Dec-2015 12:47

Palinca is always welcome and appreciated :)
 
As you know, different http headers can have different values. Some can be empty. Some must have a particular value. Some are integer values. Some have parameters.
 
Content types must be something according to the rule
Content-Type := type "/" subtype *[";" parameter]
type := "application" / "audio" / "image" / "message" / "multipart" / "text" / "video" / x-token
x-token :=
You cannot pass 'null' or an empty value etc, where the string 'null' may be valid for other headers.
 
I am/was trying to come up with a solution that I could use for various headers, without having a type for each header (<-- this is also an option).
 

Posted by agent_008_nl on 15-Dec-2015 13:38

Ok so all this is solved. Rules engines all nonsense, Palinca will come to rescue etc. Good night you various headers! You cannot pass null!

Posted by Lars Neumeier on 16-Dec-2015 05:51

I would go with the option of implementing different HttpHeader classes, Interface, Factorypattern. It's a lot easier to write tests for this scenario.

Posted by Peter Judge on 16-Dec-2015 08:32

That's a good point about testing.
 
I have a HttpHeaderBuilder already (ie factory pattern) and this is certainly the simplest (non-over-engineered :) ) approach.
 

This thread is closed