Mixed projects in OpenEdge Studio and svn

Posted by Peter van Dam on 30-Sep-2014 04:26

Hi there,


I am wondering how others are handling the situation where you have a project that mixes stuff like the following:

- ABL code

- all kinds of scripts, including linux, jenkins, ant, etc.

- documentation

- test data

- ABL test code

We would ideally want all this stuff in a single project, since it is a single project, but we only want the ABL subdirectory contents (objects and static files) distributed to the build location. So basically we would only want the compiled objects and static objects from, say, the src directory distributed to the build location (without the src directory prefix).


How do you configure this?


Thanks,

-peter

All Replies

Posted by Mike Fechner on 30-Sep-2014 04:39

We don’t commit scripts, pf, ini etc. in the source code projects. We create simple projects (not AVM runtime) for those files.

Posted by Peter van Dam on 30-Sep-2014 04:50

So basically you create two projects for one project; an ABL project and a non-ABL project.

I thought of that too but it does not sound ideal.

Posted by Mike Fechner on 30-Sep-2014 04:55

I typically create more than a single project for the source.
 
Feels o.k. to me. Gives me a good grouping by functionality or architecture.
 

-          Our framework

-          Customer add ons to the framework

-          Backend

-          Shared code

-          Frontend

-          Legacy application

 

Posted by andrew.may on 30-Sep-2014 05:27

We have a general guideline that a dev must be able to just pull a project from source control, run a build script to do any required initialisation & then be able to run / develop it straight away.
We also want a project to be able to be built straight after being pulled from source control
 
We tend to have:

·         Source + ant build script + distributable resources + Ivy dependencies file -> all in the main project’s source control repository

·         Build script can be run on a Dev PC (for testing purposes) without any extra setup. 

·         Release + QA builds are done on a Hudson instance.

·         Hudson Jobs configured to pull the required version of the SCM repository & then run the ant build script that is in the pulled repo

·         Hudson Job config is not in SCM (I’m uneasy about this, but haven’t had the chance to change it yet)

·         Build script uses Ivy to pull in dependencies (e.g. our framework & utility libraries) from an Artifactory instance

·         Build script then compiles source, packages it up in a .pl file & produces a zip containing the .pl file, any .pl files brought in as dependencies & any distributable resources (e.g. icons / default config files)

·         If project is a library or framework project then Hudson will also publish the .pl and/or .zip files to Artifactory so that downstream jobs can fetch them later.

·         Hudson then triggers any downstream jobs

 
 
[collapse]
From: Peter van Dam [mailto:bounce-pdam@community.progress.com]
Sent: 30 September 2014 10:27
To: TU.OE.Development@community.progress.com
Subject: [Technical Users - OE Development] Mixed projects in OpenEdge Studio and svn
 
Thread created by Peter van Dam

Hi there,


I am wondering how others are handling the situation where you have a project that mixes stuff like the following:

- ABL code

- all kinds of scripts, including linux, jenkins, ant, etc.

- documentation

- test data

- ABL test code

We would ideally want all this stuff in a single project, since it is a single project, but we only want the ABL subdirectory contents (objects and static files) distributed to the build location. So basically we would only want the compiled objects and static objects from, say, the src directory distributed to the build location (without the src directory prefix).


How do you configure this?


Thanks,

-peter

Stop receiving emails on this subject.

Flag this post as spam/abuse.

[/collapse]

Posted by Peter van Dam on 30-Sep-2014 07:40

Hi Andrew,

That is the svn side of things - but my question is about the OpenEdge Architect side. Is all of this in a single OEA project or in multiple?

-peter

Posted by Jean Richert on 30-Sep-2014 07:44

Test... Please ignore

Posted by Mike Fechner on 30-Sep-2014 07:44

I see benefits in using a couple of projects. Because they may have different config:
 

-          Frontend  does not require DB connections at all, may not need legacy code in the propath

-          Backend does not require .NET assemblies – but may require legacy code in the propath (include files, triggers, etc.).

 
With only a single project, you can only test this on Jenkins (where it should also be tested).
But a developer should see as many possible errors in the IDE.

Posted by Peter van Dam on 30-Sep-2014 07:53

Right Mike you are starting to convince me.

Posted by andrew.may on 30-Sep-2014 07:59

Peter,

We have (source-project == OEA-project)

i.e. a single project in source control is a single project in OEA/PDSOE

A typical OE project might look like

<Project Root>

  • src directory
  • bin directory
  • resources directory
  • build.files directory
  • wrk directory
  • lib directory
  • RunConfigurations directory
  • build.xml file
  • ivy.xml file
  • build.properties file
  • project-specific.ini file

Where the directories are used as follows:

  • src is the source folder
  • bin is where the .r code is put when built by PDSOE/OEA
  • resources is where we put files that need to be distributed with the built project (e.g. icon files)
  • build.files is where we put files that are used by the build script (& the build artifacts are put in a subdir of this)
  • wrk is used as the working directory when running the project (to keep it tidy)
  • lib is where Ivy will put any dependencies that it pulls in (e.g. pre-built .pl files of library projects)
  • RunConfigurations is where we put the PDSOE/OEA .launch files that we use to run the project's application(s)

And the magic files are:

  • build.xml - Ant build script
  • ivy.xml - Details of Ivy dependencies
  • build.properties - properties used by build.xml
  • project-specific.ini - used when running & building the project (e,g, to ensure that fonts/colours are standardised)

Of the directories:

  • the bin, lib & wrk folders (& their contents) are excluded from source control as they are generated by the build script or at runtime.
  • The src folder is the only folder that PDSOE/OEA has marked as a "source" folder & it is set to compile into bin

All the files I mentioned are in source control

Some of this is fairly arbitrary, but it has been very popular with Devs, as they can just pull a project & instantly start working on it without any manual setup.

Posted by bronco on 30-Sep-2014 08:10

Mike has a point. However, personally I find working in multiple projects a bit of a hassle. You have to keep the config's in sync, switch back and forth betweeen the projects. If I setup something I put everything in one project, including testcode, test data, build scripts etc. Also put everything in scm (my motto: if it's not in SCM, it doesn't exist)

With Jenkins in place you can unittest/compile your code with every commit to your SCM, and there are plugins for eclipse/jenkins to make those errors visible as well. Andrew's setup works perfectly wel in my experience.

Posted by andrew.may on 30-Sep-2014 08:10

By the way I'm in agreement with Mike about splitting a large offering down into individual projects.

When I talk about a "Project", I'm meaning a self-contained bit of functionality that can logically be worked on in isolation if required.

e.g.

Each of our library modules is its own project

Our ADM framework is its own project

Our PoS application is its own project

Our Back Office application is its own project

etc...

What we do do is have all the files/resources that a project needs to be developed in (& be testable) be either in the project's source control, or be provided to it at build time by Ivy.

Posted by Mike Fechner on 30-Sep-2014 08:19

Hi Bronco, spreading the code and additional files in multiple projects does not mean it’s not all under version control. It’s typically in parallel folders.
 
Assuming you’d add other tools to Eclipse (BPM, Corticon, Java Code, .NET Code, ect.) you have to work with multiple parallel projects anyway. That’s a pretty normal thing in the Java world and .NET.
 
And for working on a set of related files on a task, I use MyLyn’s task focused UI or I navigate based on the changelist view of my SCM tool (Perforce).
 
Or I navigate using CTRL-SHIFT-R and thank god, I love CamelCased names.
 
Setting up a new workspace from the SCM tool does not take any longer for one or six projects.

Posted by andrew.may on 30-Sep-2014 08:22

Bronco,

I agree with you on keeping test code in the same project as production code & config.

But I also agree with Mike that (where it makes sense,) separating logical chunks of your architecture into separate projects makes for an easier life.

Using Ivy + Artifactory to manage our dependencies makes it easy to break down our offering into interdependent projects & still keep our setups in sync,

e.g.

If Project A defines databaseA's schema & various programs that run against it, then a change to the schema of databaseA in project A will be built into the build artifacts when built by Hudson & will be pushed up into Artifactory.

Hudson will then trigger downstream jobs to rebuild themselves.

If these downstream jobs have Ivy configs that say "I depend on the latest version of project A", then they will retrieve the new version of Project A's artifacts from Artifactory (including databaseA) & will then build themselves using these retrieved artifacts (e.g. compile source against the new schema).

Devs just have to remember to run the ivy-retrieve ant task in their local projects to pull in the correct versions of dependencies.

Posted by andrew.may on 30-Sep-2014 08:24

P.S.  I'm considering putting together a presentation on using Ivy + Ant + Artifactory to do dependency management for OE projects.

Is there any interest in this?

Posted by bronco on 30-Sep-2014 09:35

I wasn't implying that multiple projects and SCM are a contradiction.

What I meant is when just OpenEdge is involved you can go either way (1 or multiple projects). I'm working on a fairly large project now and I have seen not once that a client side object used a db reference (for example), what I do see is that the inevitable errors show up rather in Jenkins (or whatever) than in the IDE, and would so even if there were multiple projects.

And yes, mylyn is great!

Posted by Peter Judge on 30-Sep-2014 11:35

@Andrew: YES PLEASE

I'm also in the "separate chunks per module" camp.  Each of my projects looks quite similar to Andrew's layout (I also have a /db and /temp).

-- peter

Posted by Peter van Dam on 30-Sep-2014 16:57

So Andrew, you are basically putting everything belonging to the module, OE or not, in an OE project.

You are saying you can select one (or more?) directory within such an OE project that is actually treated as an ABL source directory. That must be something I have missed so far.

How do you do that?

Can you do that for more than one directory?

How do you determine where the objects of each directory go?

Until now I have only worked with build destinations (objects and static files) but that is an all-or-nothing approach that applies to the entire project.

Posted by Peter van Dam on 01-Oct-2014 02:51

Ok I found it: you can do all of this in the 'Source' tab under build. That is what I have been missing so far.

Thanks guys!

Posted by Marian Edu on 01-Oct-2014 03:07

you're welcome ;)

I do keep all kind of non oe resources in the same project out of the
build path: images, build scripts, documentation... everything that is
related to the project.

If dependencies between projects you can always add workspace folders in
propath, now that I think of maybe just adding the projects to
dependency list will update the propath with each of those project's
propath?


[collapse]On 10/01/2014 10:51 AM, Peter van Dam wrote:
>[collapse] From: Peter van Dam
> Post: RE: Mixed projects in OpenEdge Studio and svn
> Posted in: Forum
> Link: http://community.progress.com/community_groups/openedge_development/f/19/p/13149/47558.aspx#47558
>
> Ok I found it: you can do all of this in the 'Source' tab under build. That is what I have been missing so far.
> Thanks guys!
>
>
> --
> You were sent this email because you opted to receive email notifications when someone created a new thread.
>
> To unsubscribe[collapse] from:
> - ...only this thread, disable notifications at http://community.progress.com/community_groups/openedge_development/f/19/p/13149/47558.aspx#47558.
> - ...all email notifications from Progress Community, navigate to "Settings", click on the "Email" tab, then under the "Email Configuration" section, set Send Notifications to "No".
>
>


--
m.edu
keep it simple
http://www.ganimede.ro
http://ro.linkedin.com/in/marianedu
medu@ganimede.ro
mobile: +40 740 036 212
skype: marian.edu[/collapse][/collapse][/collapse]

Posted by andrew.may on 01-Oct-2014 04:36

Peter,

Although it is possible to have multiple source folders in a project, I try to avoid it where possible (it makes devs' lives harder). We do have subfolders within src, but those map on to subfolders within the produced .pl file  (e.g. packages for OO code)..

The exception to the "1 source folder" rule is where we're having to pull in uncompiled source as a dependency (e.g. ADM framework include files). In that case, we might have something like lib/myFramework/src as a source folder as well.

Posted by Peter van Dam on 02-Oct-2014 07:10

OK here is a follow up question for you guys about compiling class files.

To try out the above I have specified in the source tab a directory '@{ROOT}\common' with Build Destination '@{RCODE}\common' (which is not in my project). This works fine for procedures but breaks for classes. The directory contains a class file called 'globalhelper.cls' with the class statement 'CLASS common.globalhelper'.

So what happens: the object file becomes '@{RCODE}\common\common\globalhelper.r'.

This does not happen when the entire project build destination is used.

BTW this difference in compilation behavior for class files is biting me again and again. How do you guys handle this?

Posted by Mike Fechner on 02-Oct-2014 07:14

Peter, I simply gave up on using multiple build directories and I keep src and r-code in the same folder and I am happy man.

Posted by andrew.may on 02-Oct-2014 08:07

Peter,

In your situation, I would ideally have my common code in another project & import it as a dependency.

If I had to have the common code in one of my projects for some reason, I would do this as follows  (note that all our class packages begin with our company name to keep the namespaces tidy if we ever use 3rd party code) :

/
|-> bin/
|-> src/
| |-> myLegacyProcedure.p
| |-> mycompanyname/
|   |-> myprojectname/
|     |-> mymodulename/
|       |-> MyClass.cls
|-> src-common/
  |-> mycompanyname/
    |-> common/
      |-> MyCommonClass.cls

This layout would have src and src-common as source folders, both building to bin.

The imports would be:

using mycompanyname.common.*.
using mycompanyname.myprojectname.mymodulename.*.

Posted by Peter van Dam on 02-Oct-2014 09:32

My question is rather on the difference in compilation than on common code or directory structures.


Meaning: how do you handle compiling mixed procedure and class code in (sub)directories?

Posted by andrew.may on 02-Oct-2014 10:16

The compilation is a side-product of the structure.

All our legacy code had a flat directory structure & so /src/*.p compiles to /bin/*.r

All our new code (procedural or class-based) goes into sensibly named subfolders.

e.g. /src/mycompany/myproject/mymodule/myproc.p

and /src/mycompany/myproject/mymodule/MyClass.cls

These would compile into...

/bin/mycompany/myproject/mymodule/myproc.r

/bin/mycompany/myproject/mymodule/MyClass.r

To use this in code:

run mycompany/myproject/mymodule/myproc.p.

using mycompany.myproject.mymodule.*.
def var o as MyClass.
o = new MyClass().


It does make the procedural stuff more wordy (would be nice if we had a "using" equivalent for .p's), but the unambiguity & descriptiveness of the namespaces makes up for it in our opinion.

(N.B. we've got a 3-letter company name, if you have a longer one, consider a good abbreviation instead.)

e.g. run bcp/backoffice/orderprocessing/importXML.p  is more clear than "run importXML.p"

Posted by Peter van Dam on 03-Oct-2014 04:50

I appreciate all that, but it does not address my issue.

For now I have worked aound the issue by reverting back to default root building like Mike suggested and EXCLUDING the directories that I don't want.

This does not really address my problem, because I am ending up with a 'src' directory in my build destination (which is external to the project). Oh well.

This thread is closed