Can you help with some suggestions for improvements?
Background: The method below is called to write a buffer out to file. Before that, each field in the buffer is checked to format it among other things.
I profiled a sample where we have 64 columns over 200,000 rows to output. Therefore we loop through each field for each row.
In summary:
Due to the volume of data, are there any suggestions to improve performance?
/* code ....Loop through each field in the row buffer */
if (pFieldBuffer:buffer-value = ?) then
if (pFieldBuffer:label > "") then .....
Can you post the complete code of your method?
Here is the code:
a. Overhead to call a method takes up roughly 24% of total time. Seems high to me.
b. If conditions on booleans are 10 times faster than conditions involving strings.
206 method public override logical writeBuffer(input pHBuffer as handle):
This:
pFBuffer = ttOutStream.hbuf:buffer-field(i).
is a serious performance killer, and what's more it's redudant information you only need to figure out once.
What I generally do is go through the buffer, stick all the buffer field handles in a TT, then FOR EACH through the TT as needed. Much faster.
Something like this should already be faster (less useless DO blocks, more assign).
One more thing: if your report is not going to be more than 1 gig in size, it might prove to be a lot faster to build it using a LONGCHAR and copy the longchar to a file only once at the end with COPY-LOB than to use multiple "put stream-handl".
method public override logical writeBuffer(input pHBuffer as handle):
DEFINE VARIABLE iNumFields AS INTEGER NO-UNDO. iNumFields = pHBuffer:NUM-FIELDS. if (pHBuffer:available) then do: for each ttOutStream no-lock : vCharBuffer = "". ttOutStream.hbuf:buffer-copy(pHBuffer). do i = 1 to iNumFields: pFBuffer = ttOutStream.hbuf:buffer-field(i). vReportValue = if (pFBuffer:label > "") then /* * if formatter present: right now we label attribute * to store it. For performance reasons we choose to use * the many attributes available to pass info * around for the field */ DYNAMIC-INVOKE(right-trim( "com.ifdsgroup.ifast.extract." + pFBuffer:label /* contains formatter*/), "format", input pFBuffer:buffer-value, input pFBuffer:format) else (if (pFBuffer:buffer-value = ?) then (if pFBuffer:data-type = "date" then "00000000" else fill(" ",integer(pFBuffer:width-chars))) else pFBuffer:string-value()).vCharBuffer = vCharbuffer + vReportValue + ttOutStream.columnDelimiter.
end. /* do i = */ /* presumes that the delimiter can not be part of the data (we'd have to use a substring instead if so) */ vCharBuffer = right-trim(vCharBuffer, ttOutStream.columnDelimiter). /* the higher CASE configurations take over if set */ if ttOutStream.upperCase then vCharBuffer = CAPS(vCharBuffer). put stream-handle ttOutStream.hStream UNFORMATTED vCharBuffer skip. end. /* for each*/ end. /* if available */ end method.What I generally do is go through the buffer, stick all the buffer field handles in a TT, then FOR EACH through the TT as needed. Much faster.
Did you test if the temp-table performs better than an array of the field handles (that would surprise me).
mikefe wrote:
What I generally do is go through the buffer, stick all the buffer field handles in a TT, then FOR EACH through the TT as needed. Much faster.
Did you test if the temp-table performs better than an array of the field handles (that would surprise me).
From an array of field handles? No - and I'd expect using an array of field handles to be faster still, albeit less flexible.
He's getting his buffer-field handles from the buffer handle itself though, and that's what I think is partly affecting his performance.
Thanks .... great work.