When the whole data cluster is added to RM chain?

first block in cluster has the chain pointers to next and prev cluster. these pointers are 8 bytes each.

the fact that the clusters are all chained together is what makes it possible to delete all the rows or index entries instantly. we just link the chain of clusters onto the area free list.

regards, knower of even more obscure things.
gus

I’d tell you a UDP joke, but you might not get it.



> On Jul 24, 2015, at 1:10 PM, George Potemkin wrote:
>
>[collapse] From: George Potemkin
> Post: When the whole data cluster is added to RM chain?
> Posted in: Forum
> Link: https://community.progress.com/community_groups/openedge_rdbms/f/18/t/19318.aspx
>
> Area: RPB = 16, Cluster Size = 512Chanalys shown the following for a table's RM chain:
> LIST OF RM CHAIN BLOCKS
> free# free
> dbkeyspaceslotshold
> 644207687917120
> 644207848124160
> 644208008124160
> ...
> 644218408124160
> 644218568124160
> 644218728108160
> 1265497647850
> 1265465674260
> 1265520039140
>
> First block (dbkey 64420768) at the chain's header stores 4 records (= RPB - free slots) but the next blocks are absolutely empty: no records, no recid holders.Dbkey 64421872 is the last block in data cluster. That is why its free space is less by 16 bytes than in its predecessors.Obviously the whole data cluster was added to RM chain.Does it mean that when Progress allocate new data cluster all its blocks are already linked into a chain? Does Progress add all these blocks to RM chain just by a couple of recovery notes (RL_BKFAM and RL_BKFAB)? Or does it adds them one by one (in other words, it generates a lot of recovery note)?
> Just curious,George
>
>
> --
> You were sent this email because you opted to receive email notifications when someone created a new thread.
>
> To unsubscribe[collapse] from:
> - ...only this thread, disable notifications at https://community.progress.com/community_groups/openedge_rdbms/f/18/t/19318.aspx.
> - ...all email notifications from Progress Community, navigate to "Settings", click on the "Email" tab, then under the "Email Configuration" section, set Send Notifications to "No".
>[/collapse][/collapse]

Hello Gus,

But the blocks in RM chain are linked by the pointres in the individual block headers rather than as the parts of a cluster.

I would re-word my question: if ai scan vebose shows two notes: RL_BKFAM and RL_BKFAB does it mean any of the following possibilities:

1. a single block was added to a chain's header;

as well as

2. the whole data cluster was added to a chain's header.

Regards,
George

More details:

RECORD BLOCK SUMMARY FOR AREA "Table Area" : 9,16;512
-------------------------------------------------------
                               -Record Size (B)-   ---Fragments--- Scatter
Table        Records    Size   Min   Max  Mean        Count Factor  Factor
PUB.table   35781328   25.3G   450 13519   758     43309970    1.2     1.0

1378593 block(s) found in the RM chain of Table object 362
4025852 RM block(s) found in the storage area.

Toss Limit was set to 800 bytes and right now I'm investigating why RM chain is still growing.

> less RM chain “housekeeping” means better performance

I though the same but ai scan statistics shown that the number of recovery notes related to RM chain “housekeeping” is just a small percentage of the total number of recovery notes in the same area and it was less than 1% of notes per the whole database.

Recovery notes related to a chain “housekeeping” (what I found in the tests):

add to top     RL_BKFAM (+ RL_BKFAB)
add to bottom  RL_BKMBOT (+ RL_BKBBOT,RL_BK2EB)
top to end     RL_BK2EM (+ RL_BK2EB,RL_BK2EB)
rm from chain  RL_BKFRM (+ RL_BKFRB)
extend db      RL_XTDB (+ RL_BKMAKE,RL_BKFAM,RL_BKOBJHWM,RL_BKHWM,RL_BKCHAIN_NEXT)

First note in the list applies to a object block.

Notes in the brackets are about data/index block changes.

Regards,
George

I'm going to compare the RM chain with one we will get a day later. How may blocks were added to RM chains, how many blocks were moved from a header to a chain's tail, how many blocks were updated in the middle of the chain. I hope this information will shed a light on the chain growth.

Regards,
George

add to chain at allocation time.

my memory may be defective, but there needs to be notes to take the cluster off the area free chain, or notes to raise the hwm and format each block in the cluster, plus notes to add the cluster to the table or index (a note for the block that points to the first block and plus a note for the pointer to the last block plus a note for the object's chain header to point to the last block).

or thereabouts.

regards,
gus, knower of obscure things.


I’d tell you a UDP joke, but you might not get it.



> On Jul 24, 2015, at 1:55 PM, Richard Banville wrote:
>
> When the whole data cluster is added to RM chain?

Example of raising HWM:

Code                Dbkey    UpdCtr  Block type
RL_XTDB                64     24192  Area Control Object (ACO) object block
RL_BKMAKE       335970304         0  New block added above prev HMV
RL_BKFAM            16384  88092699  Table's object block
RL_BKOBJHWM         16384  88092700  Table's object block
RL_BKHWM               64     24193  ACO object block
RL_BKCHAIN_NEXT 335970272        67  ?
RL_RMCR         335970304         1  New block
RL_BKFAB        335970304         2  New block
RL_BKFAM            16384  88092701  Table's object block
RL_RMNXTF       329096704       238  ?

Regards,
George

First of all, the correction to my previous post: some time ago the toss limit was changed to 1000 bytes (compared to 758 bytes - the average rec size).

I got new chanalys created a day after the previous one:
Date: Thu Jul 23 13:48:00 2015
Date: Fri Jul 24 13:48:00 2015

462 blocks - it's increase of RM chain length during one day.
2559 blocks were added to chain top (dbkeys from 64421888 to 64462832).
1006 blocks were moved to chain bot (dbkeys from 64389120 to 64462080).

Note: once an RM block is moved to the bottom of RM chain, the block will stay here forever if RM chain is constantly growing - to appeal a sentence is not allowed!

Only 6 blocks moved to the bottom contain recid holders (1 or 2 holds). In other words, almost all blocks were moved to the chain bottom after adding new records.
These blocks have in average 1250 bytes of free space (min: 1 byte, max: 6022 bytes).
These blocks have in average 5.7 free slots (min: 1, max: 12).
These blocks store in average 10.3 records (min: 4, max: 15).
Average record size is 691.20 bytes

Engine Crew Monograph No. 2: Space: The Final Frontier says:
If the block is not on the RM chain, has unused record slots, and has at least as much free space as the free space constant, it is added to the RM chain. If the block has more than 4 unused slots, it is added to the front of the chain. Otherwise it is added to the end of the chain.

I'm not sure what are the rules to move a block that is already on the RM chain to the end of the chain but, IMHO, these rules work inefficiently.

In this case a long RM chain can be fixed only offline (using dbrpr/1. Database Scan Menu/8. Rebuild RM Chain) but it will grow again. Toss/create limits, RecSpaceSearchDepth will not help.

Update (after a cup of coffee):

402 of 1006 blocks at the chain's bottom have free space less than toss limit
The blocks with space higher than the current toss limit have in average 5.7 free slots (min: 1, max: 12)

I has loaded first 1,048,575 rows of RM chain analysis in Excel.
Average free space is 563 bytes.
1,002,457 blocks on RM chain have free space less than the current toss limit (1000 bytes).
109,244 blocks on RM chain have free space less than the default toss limit (300 bytes).

I have rather old (Feb 23, 2015) ai scans for this database but they can give an idea about its typical activity in the area (it store only one table). BTW, toss limit had the default value (300 bytes) at this time.

Statistics of recovery notes in area 9 for two 15 min interval ai scans:

Type    Count1 Count2 Comment
RL_RMCHG 21563  28491 Change a record (fragment)
RL_RMCR   5926   8186 Create a record (fragment) or recid holder
RL_RMDEL  4852   6810 Delete a record (fragment) or recid holder
RL_RMNXTF  221    343 Insert next fragment pointer in rm
RL_BKFRB    96    108 rm from chain
RL_BKFRM    96    108 rm from chain
RL_BK2EB    40     61 top to bottom or add to bottom
RL_BKBBOT   30     33 add to bottom
RL_BKMBOT   30     33 add to bottom
RL_BK2EM     5     14 top to end

So the records are very often updated after their creation.

Engine Crew Monograph No. 2: Space: The Final Frontier says:
If the block is already on the RM chain, but now has less free space than the free space constant, or all the record slots are filled, then the block should be removed from the RM chain. But, since the RM chain is linked only in the forward direction, it can only be removed if it is the first block on the chain. If it is not the first block, it will be left in the chain. It will be removed later while searching for record space, if it should become the first block on the chain during the search.

It's not a surprise that the blocks with low free space are not removed from the chain. But they do not have the chances to be removed from the chain because the chain is growing permanently.

Regards,
George