Need fast temp-table create/fill

Posted by Etienne Begin on 12-Jan-2017 15:39

OE10.2B08 HP-UX 64i

I have about 1.7M records to load in a TT.  It takes about 14 minutes to fill the TT with a Fill() or for each with buffer-copy. That's about 2K recs per second.

Reading the 1.7M recs table takes 30 seconds.

I can speed it up by not copying the fields I don't need.  But I'd rather not do this.

This seems "slow" to me, but I could be wrong.  My -T is pointing to a good hard disk, is there anything I should look at to improve the fill performance of a temp-table ?

All Replies

Posted by Rick Terrell on 12-Jan-2017 15:50

Take a look at the -Bt (# TT buffers) and see if bumping that up from default helps.

Posted by Mike Fechner on 12-Jan-2017 15:58
Try to set -tmpbsize to 8 as well.

documentation.progress.com/.../index.html

Sent from Nine

Von: Rick Terrell <bounce-Rick_Terrell@community.progress.com>
Gesendet: 12.01.2017 10:51 nachm.
An: TU.OE.Development@community.progress.com
Betreff: RE: [Technical Users - OE Development] Need fast temp-table create/fill

Update from Progress Community
Rick Terrell

Take a look at the -Bt (# TT buffers) and see if bumping that up from default helps.

View online

 

You received this notification because you subscribed to the forum.  To unsubscribe from only this thread, go here.

Flag this post as spam/abuse.

Posted by Etienne Begin on 13-Jan-2017 10:43

tmpbsize is 8 already, but the -Bt is not increased, as this is a one time process (fill once, do work and exit).  I just got a new drive on the server will see if that makes a difference.

Posted by Rob Fitzpatrick on 13-Jan-2017 10:48

> fill once, do work and exit

If you are going to exit the session after doing the work then it isn't going to hold onto temp-table buffer memory indefinitely.  

Why not increase -Bt so the temp-table can be memory-resident?  It should improve performance significantly if you eliminate the disk I/O, more so than moving disk I/O to a better disk.

Posted by Tim Kuehn on 13-Jan-2017 10:51

Or put -T (and the DBI files) on a RAM disk.

Posted by Etienne Begin on 13-Jan-2017 12:08

Walking the whole table after fill takes less than a minute, not a problem really.  Also, I don't care if a percentage of the table is in memory when reads from disk are fast.

Posted by Etienne Begin on 13-Jan-2017 12:11

Time, I have not thought about the RAM disk.  Good idea I'll check with the datacenter if this can be done without too much cost.

This thread is closed