Recommendations for working with large projects in PDS

Posted by Patrick Tingen on 21-Jan-2016 09:03

We are transforming our way of working from our old home-grown versioning system in the classic editor towards working with GIT and PDS (11.5). Working with PDS is already done, but now we could use some useful insights in how to configure things to make it work smoothly. 

First tests show that once we work with GIT and we check out the source (> 50,000 files), PDS becomes very slow. Opening folders takes forever, which has probably to do with the project's size since small project work like a charm. To make things go smooth, we have already done:

  • Use a local workspace to avoid network traffic
  • Unlinked the editor with the project explorer
  • Disabled automatic builds

The docs state:

"For large applications with multiple modules, it is usually best to create separate projects. You must have separate projects for modules that have a different PROPATH or that require different database connections."

But I would like to have all code in one module since changes often require sources of multiple modules to be changed, which would creating and maintaining projects not straight forward. 

Are there any other tips?

All Replies

Posted by GregHiggins on 21-Jan-2016 09:19

I would consider making each top level folder a separate project. That way you maintain the source path integrity with smaller project folders.

Posted by James Palmer on 21-Jan-2016 09:19

Are all source folders within the workspace or are you using linked folders? I've found the latter incredibly slow!

Posted by Tim Kuehn on 21-Jan-2016 09:28

[quote user="Patrick Tingen"] I would like to have all code in one module since changes often require sources of multiple modules to be changed, which would creating and maintaining projects not straight forward.  [/quote]

Projects are just another way of dividing up the code in PDS - the can be stand alone or they can be inter-related with other projects, so I'd concur with Greg wrt making each top level directory a project. This'll divide the code up where PDS is concerned while retaining your "one git code base" requirement.

Some things to look out for - set each project's propaths to the required mapping between the project workspaces, and  mark each project ".project" and related files as "forget" in git so they don't become part of the SCMS code base.

Once that's done you should be good to go!

Posted by Patrick Tingen on 21-Jan-2016 09:28

Yes, we found that too, so all source folders are within the workspace, which is local. And in my case on an SSD disc

Posted by GregHiggins on 21-Jan-2016 09:36

In my current project, starting with one huge sources collection, we factored out into separate projects all Images, Assemblies, Scripts, Install, DB and Work folders. They became core projects CORE-ABL, CORE-Images CORE-Scripts CORE-Install CORE-DB CORE-WORK.  Then, overtime, newer projects were added which had more focused content. A Standard Library, a lite-weight framework, a core-cloud integration module, a Cloud Gateway project, a Generic Cloud project, specific Cloud projects and cloud server projects.  

Posted by Tim Kuehn on 21-Jan-2016 09:53

[quote user="Patrick Tingen"] Yes, we found that too, so all source folders are within the workspace, which is local. And inmy case on an SSD disc [/quote]

I generally go with one workspace per set of projects and have the projects in their own directory tree not under the workspace.

I do this because I've had workspaces get corrupted, and its a lot easier to recover from this condition if the projects aren't in the same directory tree . All I need to do is create a new workspace, configure things, import the app projects, and I'm back in business.

I also on occasion need the same project in multiple workspaces and this is easier to do if the project is not in a  workspace.

Where you put your projects is a personal preference and what works for you.

Posted by Mike Fechner on 21-Jan-2016 12:10

[quote user="Tim Kuehn"]

and  mark each project ".project" and related files as "forget" in git so they don't become part of the SCMS code base.

[/quote]

Tim, why exactly are you excluding those valuable config files?

SCM is not "source code management". SCM is "software configuration management". 

The .project file gives the project the name.

The .propath file defines the PROPATH and cross project PROPATH is dependent on the project name stored in the .project file. I also export the project configuration (project.xml) and keep that in the SCM tool.

That's important part of the configuration and I want to have that versioned nicely. I always put .project, .propath and .dbconnection under SCM.

Posted by Marian Edu on 21-Jan-2016 12:25

Fully agree with Mike on that point, guess by project configuration he meant databases configuration export... there is no point in having everyone recreating the same configuration when they check-out.

For the PROPATH you might think this is specific to each developer's workspace but that's only true if you use external resources (like full OS path), the proper way is to use project references and workspace directories instead for anything that it's outside of the project itself.

Having separate projects per modules makes a lot of sense to me, that way you can try to isolate them and build dependencies as required... now, if every module depends on all others only means the break-down is somehow wrong but at least is a start and will give you an usable development environment.

Posted by Tim Kuehn on 21-Jan-2016 12:25

Because the end goal of the original question is to get to a place where PDS can be used w/out the tool getting bogged down by the size of the code base. Splitting the application into projects is one way to accomplish that goal, but it is also an artificial division of the code base that may or may not reflect the target compile / deployment environment. If that division is not appropriate for the fully assembled code base, then .project and .propath files should be for local use only and shouldn't be part of the final system. If these files fit in with environment from development to deployment, then OP is certainly free to add those files to his code base and proceed accordingly.

Posted by Mike Fechner on 21-Jan-2016 12:29

And you prefer that each team member needs to fiddle around with PROPATH? I don't.

Posted by Matt Baker on 21-Jan-2016 12:36

 
Why would you forget the .project file ?  If you do this, then the next person in line has to recreate the project.  Include all the .set files; including .propath, .classpath, .dbconnection .  Without this you don’t have an eclipse project and you’re making more work for others.
 
 

Posted by Peter Judge on 21-Jan-2016 13:30

I generally go with one workspace per set of projects and have the projects in their own directory tree not under the workspace.

100% recommend this approach. Keep your projects where they make sense to you.
 
This also allows you to have a project in multiple workspaces (for instance, I have a 'core' project that's in at least 4 workspaces).
 
 

Posted by Tim Kuehn on 21-Jan-2016 13:45

[quote user="Peter Judge"] (for instance, I have a 'core' project that's in at least 4 workspaces).[/quote]

I have a "how does this work" project that I use as a free-fire zone for figuring things out and I don't want my messing around to be anywhere near the application I'm working on. When I need it, I'll add the project to the current workspace, resolve whatever it is I'm stuck on, use that new understanding to proceed with the application I'm working on, and then go on from there. 

Works quite the treat! 

Posted by Ramadevi Dhavala on 22-Jan-2016 05:16

Hi,

Few more things we can consider while working with large setup.

• If the work space is very large, default heap setting is mostly not sufficient. You can always edit <DLC>/oeide/eclipse/eclipse.ini file to the required settings based on your operating system, to achieve the better performance.

• 64bit PDSOE often gives the better performance when compared with 32 bit PDSOE. An apparent performance improvement is observed at most of the areas like build, compile, initializing PDSOE etc. Advantageous for memory intensive operations.

Thanks&Regards,

Rama

This thread is closed