The secrets of Master Block

Posted by George Potemkin on 10-Apr-2019 09:04

ince V10.1C the master block in Progress database contains at least 95 fields including three fields that are not reported by dbrpr utility. Some of the fields are obsolete. There are the fields that were removed from master block (like mb_dsrCRArea that existed only in V10.0A). Other fields match the ones in _MstrBlk and _DbStatus tables - so they are documented. But there are a lot of the fields in master blocks that are not documented. Below I have attached the masterblock.txt file with the list of the master block’s fields with their description. Any additional information is welcomed.

Also I attached the masterblock.sh script and the masterblock.p program. Both can read the fields of master block from disk. Both can be used for database versions from 10.0A to 12.0.

masterblock.sh dbname [field …]
If you will not specify the list of the master block’s fields the script will report all of them. The “mb_” prefix is not mandatory in the field’s names.

Or run the masterblock.p program and choose a .db file. The program will add the dump of master block to the masterblock.dump.txt file that can be opened in Excel. So you can check how the fields in master blocks are changing over the time.

Update: masterblock.sh is replaced by new version that fixes the issue on AIX.

[View:/cfs-file/__key/communityserver-discussions-components-files/18/5355.masterblock.txt:320:240]

[View:/cfs-file/__key/communityserver-discussions-components-files/18/8078.masterblock.sh:320:240]

[View:/cfs-file/__key/communityserver-discussions-components-files/18/8078.masterblock.p:320:240]

All Replies

Posted by ChUIMonster on 18-Apr-2019 20:13

George,

I didn't see anything obvious but you are more eagle-eyed than I --  is there anything in there that tells you if a replication enabled db is a source or a target vs simply being enabled?

Unfortunately the _repl-server VST vanishes at runtime if the replication server is not actually running :(

Posted by Paul Koufalis on 19-Apr-2019 00:59

I tried this on 3 repl-enabled 11.7.2 databases on AIX and mb_replEnabled returns 1 on the src and 66 on the target. I have no idea if that is definitive or not.

Posted by Paul Koufalis on 19-Apr-2019 00:59

I tried this on 3 repl-enabled 11.7.2 databases on AIX and mb_replEnabled returns 1 on the src and 66 on the target. I have no idea if that is definitive or not.

Posted by George Potemkin on 19-Apr-2019 03:49

mb_replEnabled:

0 = Replication is not enabled
1 = Source
2 = Target

Posted by George Potemkin on 19-Apr-2019 07:55

Other differences between the master block’s fields in source and target databases:

mb_repl* fields (except mb_replEnabled) are not zeroes only in source databases.

mb_maxArgnclts is 32000 in target databases. In most cases it’s zero in source databases but for one source db it was 5001 instead of 0.

mb_dsrCRAISeq is 0 if replcation is not enabled, 1 in target databases and 2 or higher in source databases.

mb_ai.* fields are zeroes if after-image logging is not enabled. In most cases it’s true for target databases.

Posted by Paul Koufalis on 19-Apr-2019 12:25

[mention:ae2ea2f6412743fc8be36c522f414ef0:e9ed411860ed4f2ba0265705b8793d05] any idea why I am seeing 66?

Posted by Paul Koufalis on 19-Apr-2019 12:36

Same source database, AIX 6.1, OE 11.7.2:

mb_dsrCRAISeq seems to change at every execution of masterblock.sh:

$ ./masterblock.sh db  2>/dev/null | grep -e Argn -e repl -e CRA

RUN 1:

mb_replai.aibegin     ERROR (seconds since Jan 01, 1970)

mb_replai.ainew       ERROR (seconds since Jan 01, 1970)

mb_replai.aigennbr    2

mb_replai.aiopen      1555457031 (seconds since Jan 01, 1970)

mb_repllstmod         ERROR (seconds since Jan 01, 1970)

mb_replupdctr         87

mb_replEnabled        1

mb_replRET            ERROR

mb_maxArgnclts        ERROR

mb_replFlags          ERROR

mb_dsrCRAISeq         92

RUN 2:

mb_replai.aibegin     1555456888 (seconds since Jan 01, 1970)

mb_replai.ainew       1555457031 (seconds since Jan 01, 1970)

mb_replai.aigennbr    2

mb_replai.aiopen      1555457031 (seconds since Jan 01, 1970)

mb_repllstmod         1555435515 (seconds since Jan 01, 1970)

mb_replupdctr         0

mb_replEnabled        1

mb_replRET            1555456963

mb_maxArgnclts        ERROR

mb_replFlags          0x00000000

mb_dsrCRAISeq         0

RUN 3:

mb_replai.aibegin     ERROR (seconds since Jan 01, 1970)

mb_replai.ainew       ERROR (seconds since Jan 01, 1970)

mb_replai.aigennbr    2

mb_replai.aiopen      1555457031 (seconds since Jan 01, 1970)

mb_repllstmod         1555435515 (seconds since Jan 01, 1970)

mb_replupdctr         ERROR

mb_replEnabled        1

mb_replRET            ERROR

mb_maxArgnclts        0

mb_replFlags          0x00000000

mb_dsrCRAISeq         0

Posted by Paul Koufalis on 19-Apr-2019 12:40

And running against the target, again AIX 6.1 OE 11.7.2:

RUN 1:

mb_replai.aibegin     ERROR (seconds since Jan 01, 1970)

mb_replai.ainew       ERROR (seconds since Jan 01, 1970)

mb_replai.aigennbr    ERROR

mb_replai.aiopen      ERROR (seconds since Jan 01, 1970)

mb_repllstmod         ERROR (seconds since Jan 01, 1970)

mb_replupdctr         ERROR

mb_replEnabled        66

mb_replRET            1555456768

mb_maxArgnclts        32000

mb_replFlags          ERROR

mb_dsrCRAISeq         ERROR

RUN 2:

mb_replai.aibegin     1555456768 (seconds since Jan 01, 1970)

mb_replai.ainew       1555457024 (seconds since Jan 01, 1970)

mb_replai.aigennbr    0

mb_replai.aiopen      1555457024 (seconds since Jan 01, 1970)

mb_repllstmod         1555435264 (seconds since Jan 01, 1970)

mb_replupdctr         0

mb_replEnabled        66

mb_replRET            ERROR

mb_maxArgnclts        ERROR

mb_replFlags          0x00000000

mb_dsrCRAISeq         1

RUN 3:

mb_replai.aibegin     1555456768 (seconds since Jan 01, 1970)

mb_replai.ainew       1555457024 (seconds since Jan 01, 1970)

mb_replai.aigennbr    0

mb_replai.aiopen      1555457024 (seconds since Jan 01, 1970)

mb_repllstmod         1555435264 (seconds since Jan 01, 1970)

mb_replupdctr         0

mb_replEnabled        66

mb_replRET            ERROR

mb_maxArgnclts        ERROR

mb_replFlags          ERROR

mb_dsrCRAISeq         ERROR

Posted by George Potemkin on 19-Apr-2019 12:52

Paul,

Most likely masterblock.sh works incorrectly on AIX. I did not test it there.
Use masterblock.p or 'viewdbblock.sh dbname 6 64' (text is updated)

[View:/cfs-file/__key/communityserver-discussions-components-files/18/viewdbblock.sh:320:240]

Posted by Paul Koufalis on 19-Apr-2019 13:05

From masterblock.p No errors but still not what you expected.   FYI: note the space in "mb_ replFlags" in the dump.txt file.

Source:

mb_replEnabled 1

mb_replRET 1555442766

mb_maxArgnclts 0

mb_ replFlags 0

mb_dsrCRAISeq 91

Target:

mb_replEnabled 66

mb_replRET 1555442766

mb_maxArgnclts 32000

mb_ replFlags 0

mb_dsrCRAISeq 1

Posted by Paul Koufalis on 19-Apr-2019 13:08

Hmmm....

$ ./viewdbblock.sh db 64

viewdbblock.sh: wrong area number (64).

Posted by George Potemkin on 19-Apr-2019 13:13

Today is not my day. ;-(

It should be 'viewdbblock.sh db 6 64' where 6 is the area number and 64 is dbkey of master block provided db blocksize is 8K.

Posted by Paul Koufalis on 19-Apr-2019 13:18

viewdbblock.sh db works:

Source:

    mb_replEnabled:       0x01               1

    mb_replRET:           0x5cb62c4e         1555442766

    mb_maxArgnclts:       0x00000000         0

    mb_replFlags:         0x00000000         0

    mb_dsrCRAISeq:        0x0000005d         93

Target:

    mb_replEnabled:       0x42               66

    mb_replRET:           0x5cb62c4e         1555442766

    mb_maxArgnclts:       0x00007d00         32000

    mb_replFlags:         0x00000000         0

    mb_dsrCRAISeq:        0x00000001         1

Posted by George Potemkin on 19-Apr-2019 13:22

I still don't have an idea why the target's mb_replEnabled is 66 on AIX. Not all secrets of master blocks are revealed!

Posted by Paul Koufalis on 19-Apr-2019 13:22

viewdbblock.sh db 6 64 returns the same result as without the "6 64" at the end.

Posted by George Potemkin on 19-Apr-2019 13:29

Paul, can you send me the full results of both scripts and masterblock.p? I would like to compare the results on AIX with my collaction of master block's dumps.

Posted by Dapeng Wu on 19-Apr-2019 13:30

mb_replEnabled can have multiple flags in the field.

For a replication source database, it will have the SOURCE bit set (bit value of 1); and for the target database, it will have TARGE bit (bit value of 2) set.

In some rare cases, there could be some other bits set to the field. For example, if a target is in the pre-transition state, it will also have the PRE-TRANSITION bit (bit value of 64) set. So mb_replEnabled of 66 means it's a TARGET database, and it's in PRE-TRANSITION state (66 = 2 + 64).

If you just need to find out if a database is SOURCE or TARGET, you just need to check if bit (1) or bit (2) is set in the field -- note that we're talking about bitwise operations here.

Posted by ChUIMonster on 19-Apr-2019 13:34

I don't suppose that the _Block VST will ever be of any use in this endeavor?

Posted by Paul Koufalis on 19-Apr-2019 13:34

Sending by email.

Posted by gus bjorklund on 19-Apr-2019 14:12

> On Apr 19, 2019, at 9:35 AM, ChUIMonster wrote:

>

> I don't suppose that the _Block VST will ever be of any use in this endeavor?

>

>

>

in what endeavor?

-gus

Posted by George Potemkin on 19-Apr-2019 16:30

I wrote:

> mb_repl* fields (except mb_replEnabled) are not zeroes only in source databases.

Paul's case shows: these fields are identical (non-zeroed) on source and target when a target is in the pre-transition state.

Posted by Paul Koufalis on 19-Apr-2019 17:14

The databases are in normal processing, though it is possible that no changes have gone through since the DB was started (this is a test system).

Posted by George Potemkin on 19-Apr-2019 17:35

> mb_replRET            ERROR

masterblock.sh seems to report the errors occasionally on AIX i.e. if you will re-run the script it may or may not issue the error again for the same database.

  dd if=$2 bs=1 skip=$Offset count=4 2>/dev/null | \
  od -d | \
  awk 'NF<3 {print "ERROR"

Correct behaviour:

echo 1234 | dd bs=1 skip=0 count=4 2>/dev/null | od
0000000 031061 032063

IMHO, it's the issue of dd command: its output can sporadically get truncated. I saw the similar issue on Linux but with two dd command in pipe. It was easy to reproduce in test. So it's not a bug of the script. Unforunately I don't know a workaround. Progress is much more reliable than shell scripting. Masterblock.p is more reliable than masterblock.sh. Though the current version of masterblock.sh seems to work reliably on Linux and HP-UX.

Posted by Paul Koufalis on 19-Apr-2019 17:37

The errors were definitely intermittent.

Posted by gus bjorklund on 19-Apr-2019 17:54

man page for dd on AIX says:

"The block size values specified with the bs flag must always be a multiple of the physical block size for the media being used"

but you can use od thusly, without dd:

od -d -j $Offset -N 4 filename=

Posted by George Potemkin on 19-Apr-2019 17:59

Thanks, Gus. I'll give it a try.

Posted by George Potemkin on 22-Apr-2019 15:40

Removing the usage of dd command did fix the issue with masterblock.sh on AIX. Thanks, Gus.

I uploaded new version of the script in the first post (but the forum engine changed the names of the attached files).

Additional note:

proutil -C disablesitereplication does not seem to zero all fields (mb_replai.*, mb_maxArgnclts, mb_lastLogOpSeq) that were set when the replication was enabled. So mb_replEnabled is the only reliable flag to check if the replication is enabled.

Posted by George Potemkin on 24-Apr-2019 13:34

> I don't suppose that the _Block VST will ever be of any use in this endeavor?

Would it be useful a 4GL program that can read and parse the database blocks from disk?
I think it could be used:
1. as a quick check after the 1124’s: “Wrong dbkey in block”;
2. to identify the objects that own the blocks reported by after-image scan or by Status: Blocked Clients in promon or by _Connect-Wait* or by Status: Buffer Lock Queue.

A “plugin” to convert dbkeys in after -image scan into the object’s IDs and/or object’s names can significantly slow down the scan of AI files. Would the new information worth the time lost during the scan?

It’s not hard to write a program that would parse only the block headers. Though my first attempt was failed due to a Progress bug: sometimes Progress read data from a wrong offset when the offset was larger than 2GB. I hope the bug is fixed now. Also program could optionally parse the context of whole block using the same trick as the viewdbblock script.

Posted by George Potemkin on 06-Jun-2019 09:52

There are two new fields in V12.0 (as of Fri Feb 22 19:02:39 EST 2019):

085 0x0055   1 mb_bkincr_HIGH       0x0000             0x%04hx    %d    12.0
084 0x0054   1 mb_buseq_HIGH        0x0000             0x%04hx    %d    12.0

Extension of mb_bkincr / mb_buseq?

Both fields are reported by dbrpr as two bytes but in fact their size is only 1 byte.

Posted by Kyle Monico on 06-Jun-2019 12:27

Yes, those two fields are high byte extensions of the backup counter and backup sequence fields in the master block. The backup counter was extended to three bytes in 12.0 for all Type II storage areas using a spare byte in the headers. The master block doesn't have this spare byte in the header, so a separate, spare byte was selected. The backup sequence is the last successful backup counter value, so it also needs to hold up to three bytes. The values are pieced together whenever they are retrieved and set at runtime.

Dbrpr should display them as 1 byte, I'll file a bug to fix that.

This thread is closed