Converting from oe10.2 to oe11.5

Posted by Quintin on 17-Feb-2015 06:30

One of our clients is looking to upgrade from OpenEdge 10.2B to OpenEdge 11.5. What I would like to know is, would it be best to do a dump and load or can I do a conversion?

 

They are worried that with a conversion they won’t get all the new features to 11.5, thus thinking a dump and load will be better but I can’t see why a conversion from 10 to 11 won’t work.

 

Is this the case?

 

Awaiting your answer

 

Regards

Posted by Rob Fitzpatrick on 18-Feb-2015 11:13

More on Type II:

Performance

  • Type II areas are composed of clusters of blocks. Cluster size is 8, 64, or 512 blocks and is specified at the storage area level. Type II areas are extended a cluster at a time (rather than a block at a time as with Type I). This minimizes the number of extend operations for variable extents and makes data more contiguous on disk, improving sequential I/O.
  • In a Type I area, a storage object's data can reside in any block in the area. In Type II, a storage object comprises a chain of clusters. This allows object-based utilities (e.g. tablemove, idxmove, idxbuild record scan, etc.) to be more efficient.
  • In Type I areas, records from different tables may be interleaved within the same block. In the worst case, a scan of a 1000-record table could require 1000 RM blocks to be read into the buffer pool, potentially forcing other useful data to be paged out. Whereas in Type II areas, each block (and all blocks in that block’s cluster) contains data from only one storage object. This increases buffer hit percentage and the ratio of logical to physical I/O versus Type I and improves application performance.
  • A full table scan of a table in a Type II area can be performed via ABL without an index, using the TABLE-SCAN record phrase. This is the fastest way to scan a full table (although the records are not ordered).

Reliability

  • In Type II areas every block contains a checksum which is verified every time the block is read, meaning corruption can be detected and reported in the DB log.

Availability

  • The second phase of index rebuild (data scan/key build phase) for tables in Type II areas can be multi-threaded with the -datascanthreads n parameter (subject to certain conditions). Depending on factors such as available cores, available RAM, and area size, multi-threading can significantly reduce the overall index rebuild time, which means reduced downtime.

Flexibility

  • When using Transparent Data Encryption with Type II areas, data can be encrypted at the level of individual storage objects (tables, indexes, LOB columns). In Type I areas, you can only encrypt an entire area.
  • When using Alternate Buffer Pool with Type II areas, you can assign areas or individual storage objects to the ABP. With Type I you can only assign an entire area to ABP.
  • In Type II areas there is one RM chain (a linked list of RM blocks with available space for record storage) per table, and thus individually-tunable create and toss limits per table as well. This allows DBAs to better tune the storage to the characteristics of the data, fill blocks more efficiently, and limit record fragmentation. In Type I areas there is a single RM chain, and create and toss limits, per area.

Access to new features

  • When using OpenEdge Auditing, the audit data and index areas must be Type II.
  • When using Transparent Data Encryption, the encryption policy area must be Type II.
  • When using Multi-tenant Tables, multi-tenant tables (table partitions) must be in Type II areas.
  • When using Table Partitioning, table partitions will have to be in Type II areas.
  • If you have your data in Type II areas now and you decide to use these and other new database features that require Type II areas then you can access them immediately. If your data is still in Type I areas you won't have access to these features until you dump and load.
  • It is safe to say that all new major RDBMS enhancements will function only (or at least, function best) with Type II storage.

This is by no means a comprehensive list of Type II advantages.

In short, there is no scenario where Type I areas are more beneficial than Type II. Also, as the schema area must be Type I, you should ensure that you have no application objects stored there.

All Replies

Posted by ChUIMonster on 17-Feb-2015 08:06

They key question is -- do they already have type 2 storage areas?

If "yes" then a conversion is probably suitable. It may still be worth
doing a d&l for other reasons but the main task of quickly getting to
11.5 and having new features available can be accomplished by conv1011.

If "no" then not only are the 11.5 features unavailable but most of the
goodness of 10.2 has been unavailable.

The second question would be "what specific 11.x features do you want?"

Posted by S33 on 18-Feb-2015 09:39

Tom -- If you would, please elaborate on the "goodness" on type 2 storage areas (or give a link to an elaboration).

Thx --

Jim Shepherd

jrs@lvlsvn.com

Posted by ChUIMonster on 18-Feb-2015 10:10

In short -- type 2 storage areas are the foundation that all new
database features are built upon. Hardly any of "the good stuff" is
available for type 1 areas.

http://dbappraise.com/ppt/sos.pptx

To upgrade to oe10 or oe11 and continue running type 1 storage is
missing the point. Without them you may as well be running v9 from a
dba point of view.

Yes, you might argue that if you have a tiny database that is hardly
used there isn't much benefit. But if that is the case it is also
hardly any effort to convert. And by converting you will be ready for
the day when your company (or if you are a reseller, one of your
customers) suddenly says "oh, our db is 1TB of data and performs like a
pig wallowing in mud".

Also -- if you are asking this question it seems likely that you may
also be unaware of the need to be running after-imaging.

http://dbappraise.com/ppt/ai.pptx

Every database that has valuable data in it should be running
after-imaging. If you are not running with after-imaging you are
basically saying that you don't care if you lose your database.

Posted by S33 on 18-Feb-2015 10:55

Thanks Tom. That may take more than reading for me to digest  :-)

I am running after-imaging but I appreciate the "heads up". I only had 1 occasion to do an AI recovery and it fell apart. AI was in its infancy and the client had lost about 1 hour's worth of (manually keyed) data. The AI recovery ran about 4-5 hours before the customer made me stop it. We joked about John Henry and his hammer and how these humans were 5+ times faster than AI. But in all seriousness, I do recommend and/or insist on implementing AI at all my sites.

Posted by ChUIMonster on 18-Feb-2015 11:12

That must have been a very, very long time ago.

In modern releases after-imaging is rock-solid.

Pretty much nobody is in a position to re-key data anymore. Although all
too many managers have not realized that. Yet.

Posted by Rob Fitzpatrick on 18-Feb-2015 11:13

More on Type II:

Performance

  • Type II areas are composed of clusters of blocks. Cluster size is 8, 64, or 512 blocks and is specified at the storage area level. Type II areas are extended a cluster at a time (rather than a block at a time as with Type I). This minimizes the number of extend operations for variable extents and makes data more contiguous on disk, improving sequential I/O.
  • In a Type I area, a storage object's data can reside in any block in the area. In Type II, a storage object comprises a chain of clusters. This allows object-based utilities (e.g. tablemove, idxmove, idxbuild record scan, etc.) to be more efficient.
  • In Type I areas, records from different tables may be interleaved within the same block. In the worst case, a scan of a 1000-record table could require 1000 RM blocks to be read into the buffer pool, potentially forcing other useful data to be paged out. Whereas in Type II areas, each block (and all blocks in that block’s cluster) contains data from only one storage object. This increases buffer hit percentage and the ratio of logical to physical I/O versus Type I and improves application performance.
  • A full table scan of a table in a Type II area can be performed via ABL without an index, using the TABLE-SCAN record phrase. This is the fastest way to scan a full table (although the records are not ordered).

Reliability

  • In Type II areas every block contains a checksum which is verified every time the block is read, meaning corruption can be detected and reported in the DB log.

Availability

  • The second phase of index rebuild (data scan/key build phase) for tables in Type II areas can be multi-threaded with the -datascanthreads n parameter (subject to certain conditions). Depending on factors such as available cores, available RAM, and area size, multi-threading can significantly reduce the overall index rebuild time, which means reduced downtime.

Flexibility

  • When using Transparent Data Encryption with Type II areas, data can be encrypted at the level of individual storage objects (tables, indexes, LOB columns). In Type I areas, you can only encrypt an entire area.
  • When using Alternate Buffer Pool with Type II areas, you can assign areas or individual storage objects to the ABP. With Type I you can only assign an entire area to ABP.
  • In Type II areas there is one RM chain (a linked list of RM blocks with available space for record storage) per table, and thus individually-tunable create and toss limits per table as well. This allows DBAs to better tune the storage to the characteristics of the data, fill blocks more efficiently, and limit record fragmentation. In Type I areas there is a single RM chain, and create and toss limits, per area.

Access to new features

  • When using OpenEdge Auditing, the audit data and index areas must be Type II.
  • When using Transparent Data Encryption, the encryption policy area must be Type II.
  • When using Multi-tenant Tables, multi-tenant tables (table partitions) must be in Type II areas.
  • When using Table Partitioning, table partitions will have to be in Type II areas.
  • If you have your data in Type II areas now and you decide to use these and other new database features that require Type II areas then you can access them immediately. If your data is still in Type I areas you won't have access to these features until you dump and load.
  • It is safe to say that all new major RDBMS enhancements will function only (or at least, function best) with Type II storage.

This is by no means a comprehensive list of Type II advantages.

In short, there is no scenario where Type I areas are more beneficial than Type II. Also, as the schema area must be Type I, you should ensure that you have no application objects stored there.

Posted by Quintin on 19-Feb-2015 00:56

Thanks SS3, ChuiMonster and Rob for clearing things up for me!

I am aware of AI, I did mention this to our client but they have other means of backup they have in place and trust... Who am I to argue... I did go and have a look at their databases and it seems like only two databases out of 24 make use of Type II storage areas.  Note! This is a client looking to stay updated with the latest software but they aren’t making use of the full advantages of the latest OpenEdge.

So if they want to upgrade to 11.5 I need to tell them what needs to change for them to have to full advantage of the Latest OpenEdge otherwise there is no point in upgrading if they don’t plan on using it.

Again Thanks for the feedback and all the help!

Posted by S33 on 18-Mar-2015 09:32

Tom --

I spent a little more time in the storage optimization ppt you recommended.

Can you explain the logic behind this statement:

"Indexes should always be assigned to a storage area whose rows per block is set to 1. "

Thx in advance --

Jim Shepherd

jrs@lvlsvn.com

Posted by ChUIMonster on 19-Mar-2015 08:34

That's a topic of some debate and personal preference.

Rows per block is meaningless to indexes.

The thinking behind the recommendation is that RPB = 1 says "this storage area is for indexes, don't put anything else in it."

The other perspective is "if you accidentally put something else in that area it will probably get very big, very fast."  I can confirm that from personal experience, having done so at least once ;)  But -- that rapid growth also serves as a "canary in the coal mine" that you have an issue that needs attention...

Of all the various recommendations in that presentation that is the one that I would be least likely to spend much time worrying about.  If you think that you would rather go with rpb = 256 "just in case someone puts a table in an index area" I might raise an eyebrow but I'm not going to spend much time trying to talk you out of it.


[collapse]On 3/18/15 10:32 AM, S33 wrote:
Reply by S33

Tom --

I spent a little more time in the storage optimization ppt you recommended.

Can you explain the logic behind this statement:

"Indexes should always be assigned to a storage area whose rows per block is set to 1. "

Thx in advance --

Jim Shepherd

jrs@lvlsvn.com

Stop receiving emails on this subject.

Flag this post as spam/abuse.



-- 
Tom Bascom
603 396 4886
tom@greenfieldtech.com
[/collapse]

This thread is closed