disable / speed up automatic truncate of bi in binary load

Posted by Grant Holman on 03-May-2017 06:07

I'm doing a binary D&L of a 9.1D DB which has around 300 tables.  During the load phase I run a command line for each table as follows:

proutil myDB -C load tablex.bd build indexes -i -TB 31 -TM 32

Even though I'm running in no-integrity mode I still get this message at the end of each table load:

"The BI File is being automatically being truncated. (1526)"

This is adding 60 seconds to every load operation - so 60 x 300 is 5 hours just to keep truncating the BI, which isn't being written to anyway!

There must be a faster way to do this.  Any pointers appreciated.

All Replies

Posted by mfurgal on 03-May-2017 06:20

Add -G 0 to the command line.  That removes the 60 second pause time

Sent from my iPad

Posted by Grant Holman on 03-May-2017 06:21

Just realised its the "build indexes" clause causing this to happen.  I'll remove it and do a seperate proutil -C idxbuild all.

Posted by Rob Fitzpatrick on 03-May-2017 06:23

> ... just to keep truncating the BI, which isn't being written to anyway!

While proutil idxbuild doesn't write BI notes, proutil load does.

Posted by cjbrandt on 03-May-2017 08:32

You are loading all the tables in single user mode.

Instead try to load just the largest tables in single user mode.  

Then start the database with -i setting and load the smaller tables.  The database utilities won't need to truncate the BI after each table.

Once your database is in multiuser mode, if your 9.1D database has multiple areas, then you can be loading each area at the same time.

Posted by Grant Holman on 03-May-2017 09:02

Thanks to you all for your replies.

Posted by Grant Holman on 03-May-2017 09:12

Hi cjbrandt, is there a way to specify an area when performing an idxbuild?

The only way I can think to do this is to use the metaschema to work out what indexes are in an area, then build a text file listing them and using stdin to pass it to proutil -C idxbuild...

Posted by cjbrandt on 03-May-2017 09:23

I don't think so, but it has been too long since I worked on 9.1D to remember, what idxbuild could do then.  If this is concerning the D&L, I would just include the "bulid indexes" phrase with the proutil load command for all tables and avoid running the idxbuild.

If I recall correctly, you want to either use build indexes for every table, or for none of the tables, because the idxbuild command will scan through the entire database with a single thread when you run it.   That takes a large amount of time.

Posted by Rob Fitzpatrick on 03-May-2017 09:51

9.1D idxbuild lets you select "some" or "all".  It doesn't give all the menu choices that are in later releases, like rebuilding by area, by index status, etc.

You can enter table and index names interactively with the "some" option.  Or you might be able to build a dbname.xb file and rebuild with that.  I haven't tried it in 9.x but it's worth a shot.  See this KB for details:

http://knowledgebase.progress.com/articles/Article/P68731/p

Posted by Grant Holman on 03-May-2017 10:07

Thank cjbrandt, I'll do a comparison of both methods and report back.

Posted by Grant Holman on 03-May-2017 10:08

Cheers Rob, that KB looks interesting, I wasn't aware of this feature.

Posted by Grant Holman on 05-May-2017 08:21

cjbrandt - I found using "build indexes" as part of the binary load moderately faster than doing a separate idxbuild step.

Posted by Paul Koufalis on 05-May-2017 08:53

@grant it depends on your version of Progress. Newer versions have -TF available for idxbuild which is not available when you build indexes inline. Also from a risk-based perspective in a tight d&l window, you are better off running tabanalys and double checking your before-and-after record counts before investing the time to idxbuild.

This thread is closed