PUT-KEY-VALUE is sloooooooow

Not using an ini file would be one. In general, wherever possible, I have migrated applications away from using ini files as much as possible. In most applications you can integrate the config into the database, but DataDigger doesn't have its own DB. Have you considered just EXPORTing the temp table to a .d file? As long as you only add fields to the end of the definition it should always work on the import.

Thinking some more, maintaining a custom file in a longchar that you periodically copy-lob to disk might be quicker as well.

Hi Patrick,

Writing some many changes to an INI file or any other file is bad. 60 times a second is not even that bad. With that many read and writes you could use an ssd.

There are a couple of solutions, but a database is the best option.

With regards,

Ralph

Why not just using write-json on that temp-table?

The bad news is that I am already using an ssd. A db would be best, but as James stated, the DataDigger does not have its own database so that is no option. Writing it to disk myself might be the better option here.

Problem is that most of the time there are just a few settings that get flushed to disk. Writing the complete INI might take longer than just using PUT-KEY-VALUE on a handful of settings.

Ah, well, let's test and find out.

Try it with a memory disk, maybe it's going faster.

Try it with a memory disk, maybe it's going faster.

Some anti-virus programs can cause writes to ini files to be slow. Try changing the extension on your ini file to something other than "ini" to see if performance improves. Otherwise, there's not much that can be done to improve the speed of PUT-KEY-VALUE in OpenEdge since the real work is done by Windows function WritePrivateProfileString.

You are using -basekey INI I presume?

I tried a little test to see what's going on in PUT-KEY-VALUE:

define variable i     as integer   no-undo.
define variable v-key as character no-undo.
define variable v-val as character no-undo.

etime( yes).

do i = 1 to 1000:
  assign
    v-key = "foo" + string( i, "9999" )
    v-val = "bar" + string( i, "9999" ).

  put-key-value section "mysection" key v-key value v-val.
end.

display etime.

It took 4.71 sections to create 1000 key-value pairs.  Each call to PUT-KEY-VALUE results in eight file operations:

CreateFile
LockFile
QueryStandardInformationFile
ReadFile
WriteFile
SetEndOfFileInformationFile
UnlockFileSingle
CloseFile

Most of these operations are quick, ranging from about 30 microseconds to maybe a millisecond.  But CloseFile is by far the most expensive, at 4.41 seconds, or 93.5% of the total elapsed time.

So if you want to write a lot of keys all at once with high throughput, maybe James' idea of writing to the file with COPY-LOB would be better as you would only open and close the file once.

Just curious: have you tried using the registry rather than an ini file?  I wonder how the performance would compare.

Yes, I am using -basekey INI for the DataDigger since I would like it to be portable.

I rewrote the procedure that writes the settings so it just writes it to disk itself instead of relying on PUT-KEY-VALUE. The time to write all settings to disk is now consistent; in my testcase 30-35 msec, no matter how many settings changed.

In the old situation, writing took about 12 msec per setting, so from 3 settings or more, this way is faster.

Excellent solution now, thanks everyone.

Rob:

"Each call to PUT-KEY-VALUE results in eight file operations:"

Which tool do you use to monitor those operations?

Hi Jeff,

As Brian said, I used procmon.

I never really understoof why that INI file stuff needed to be Windows-only.

I actually thought about using INI files for a while until I discovered that unfortunate restriction.