Data Version Control

Posted by Michael Ferrin on 25-Aug-2017 13:33

Good afternoon fellow Progress peoples,

This is more of an opinion question. What processes are you guys using to use source control to track changes to maintenance type tables and data? I've seem tools like phing for PHP used with a mysql database. Is there something that others are doing specifically with Progress?

Thanks everyone!


All Replies

Posted by Mike Fechner on 26-Aug-2017 02:48

Progress Dynamics did a great job at versioning (relational) data. There’s still documentation about that in the Progress doc set.
In our SmartComponent Library framework, we have implemented a similar approach.
First, all of our data is maintained through Business Entities (around ProDatasets). And we use the XML format of the ProDatasets as our dump format. We have our own dump and load routines. There are two significant benefits over Progress .d files:
  • Export of relational data (parent/child records) in a single XML file
  • The import has no problem with records already existing when loading data
We use this for versioning data in our development and also for deploying data to our customers.
We have two different types of datasets. Many are dumped as a whole file with multiple records (languages, error messages, …) and others are dumped on a per master record set (e.g. repository objects).
We also have a mechanism to distribute deletions that way. We record deletions in the framework and depending on the dataset, we deploy those deletions to our clients as well.
We dump formatted XML files – which makes them also pretty nicely comparable in the SCM tool’s diff viewers.

Posted by Richard.Kelters on 27-Aug-2017 16:17

Hi Mike,

At our customer we've done they same as other Mike did. Export data as formatted xml; menu, authorisation, repository and more. Version control is most of the time able to merge the differences per change.

We're now actually starting to remove the data from the database and start using the xml at runtime. As this data is read only at runtime we've found no gain to read it from a database but this depends on the amount of data of course. It also helps developers not to have a repository database either shared or separetly  maintained and it solves the delete problem. Our appserver agents read a couple of hundred xml files into static temp tables and we did not have to change any code in the framework.


This thread is closed