Code pages

Posted by rbf on 12-Oct-2009 06:23

Our client uses code page 1252 (and so does our database). The cpstream parameter, however, is set to utf8 since we need that for various external systems.

At design time, everything looks fine.

However, at run time, special characters (such as ë) are not displayed correctly. This is rather strange, since they are displayed correctly at design time (in OEA).

The problem can be fixed by removing the cpstream setting, however that is not an option.

I would expect there would be another parameter for this, but we have not found it yet. In addtion, I would expect that the conversion from 1252 to utf8 to be working correctly!

Does anyone have a solution (I suspect Mike knows all about this - as usual).

All Replies

Posted by Peter Judge on 12-Oct-2009 07:26

However, at run time, special characters (such as ë) are not displayed

correctly. This is rather strange, since they are displayed correctly at

design time (in OEA).

This may be somewhat tangetial to your question, but it is possible to set OEA to different code pages (Window/Preferences/Workspace). So you might then see the funkiness in both places.

-- peter

Posted by rbf on 12-Oct-2009 07:46

pjudge wrote:

However, at run time, special characters (such as ë) are not displayed

correctly. This is rather strange, since they are displayed correctly at

design time (in OEA).

This may be somewhat tangetial to your question, but it is possible to set OEA to different code pages (Window/Preferences/Workspace). So you might then see the funkiness in both places.

-- peter


That raises an interesting question: which codepage do you set in that case? cpinternal? cpstream? Java? Does it apply to the Eclipse editor of to the Progress client or the .NET client?

I would expect the code page of the Progress client and the .NET client to be the same. Obviously that is what I need and want.

BTW the Eclipse code page is currently set to 1252 which is the same setting as cpinternal.

Posted by Matt Baker on 12-Oct-2009 10:51

I'm assuming you are using .resx files.  In this case the cpinternal is the one that matters.  The .resx files are stored in utf-8 format which is read by .net and converted to whatever the machine is running.  You can set cpinternal to utf-8 if practical or change the format that the .resx files are stored to match your cpinternal code page.  window->preferences->general->content types->text->xml->openedge visual designer resource.

Posted by Garry Hall on 12-Oct-2009 13:28

Hi Peter,

You might need to provide more information on exactly where it "looks fine" at design time, but not correct at runtime. For example, whether it looks fine in the source code, on a label in an ABL window, or in a .NET form. I was first thinking it was an issue with the encoding of the source code, but as Matt suggested, it might be the .resx file.

I would also suggest logging a call with Technical Support, with a reproducible case.

Best Regards,

Garry.

Posted by Matt Baker on 13-Oct-2009 10:49

To answer the rest of the questions. The answer somewhat depends on where the text strings are stored. In many cases you will get some stored in the .cls and some stored in the .resx files so perhaps both apply.

When working with .resx files

The .resx file is written by java whenever you save the form/user control in visual designer.  It is supposed to respect the code page specified for the .resx file type based on the content type setting in the preferences (see previous email).  This is defaulted to utf-8.  There are two things at play here.  Both the internal format of the .xml declared in the header of the .xml file and the on disk bytes where the .xml file in its string form stored in the Eclipse document is converted to bytes and written out to the file system.  This is where the content type settings is used.

Internally, both the .NET side of visual designer and the java side keep all strings as utf-16 since that is the native format of both VMs.

Communcation between .net and java sides of the visual designer use UTF-16 as well since that is native format for strings.

The .resx file is read the java side when opening a form. For the most part it is treated as a blob of characters as a single document in Eclipse.  Eclipse sends the blob of characters to the .NET side to be used by the designer, there the .NET side converts it from a blob of characters to an in memory .xml document (as a ResourceSet object).  If there are multiple .resx files they are all sent together to be used to display properties for different languages.

The .resx file is also read by the .NET side as a resource set when the form is opened.  It is read when the form is loaded to provide the strings to the designer.  These strings get passed through the AVM on each assignment in initializeComponent.

The .resx file is read at runtime by .NET code.  There is no java involved at this point.  This is more or less the same process that occurs at design time.  Each string read from the resource set is passed through the AVM statements in initializeComponent.  You'll see lines that look like this:

THIS-OBJECT:Text = resources:GetString("$this.Text").

The string is stored in .NET as it was read from the .resx file, the result of the GetString() method call is handled by the AVM bridge code, and is then assigned to the Text property of the .NET object (or whatever appropriate property be it .NET property or ABL property).  This is where cpinternal comes into play with respect to the .NET code.

So in summary. resx files are handled as UTF-8 by both java and .NET and are written out as such into the .resx file as UTF-8 encoded xml.  When they are read into .NET to build a resource set then the strings themselves are in the native format of the VM, and communcation between the .NET VM and the AVM is controlled by cpinternal which should match what is used by .NET.

When working with text stored in the .cls file

Now there is a second issue here when you are working with strings stored in .cls files.  When the .cls file is saved Eclipse encodes the characters in the document and saves them into the file using whatever content type encoding is setup for the file or the workspace.  By default this is cp1252.  When this file is compiled (or run directly..it doesn't matter) prowin32.exe reads the bytes in the .cls file using the encoding specified by -cpstream to convert them into characters.

The value of -cpstream needs to match the encoding used to save the file.  If not, then the ABL compiler can misinterpret characters and put the wrong values into the r-code.    If you are using -cpstream utf8 then you need to make sure all your .cls files are encoded in utf8 format.  You can make sure Eclipse reads and saves the files using the correct encoding by looking at the content type settings at either window->preferences->general->workspace or window->preferences->general->content types.

Posted by rbf on 13-Oct-2009 17:22

Thanks Matthew, I will look into this and keep you all posted.

Posted by Admin on 22-Oct-2009 04:17

Matt, I remember there was a procedure to set the code page for an Eclipse project to ibm850. IBM850 is not supported by default. Something with using a newer Eclipse release or different JVM version.

Can you help?

Posted by Matt Baker on 22-Oct-2009 07:07

The supported code pages are ased on the JVM.  You would need to use a different JVM or find the code pages for the JVM you are using.

Posted by Admin on 22-Oct-2009 07:26

The supported code pages are ased on the JVM. 

Do you have a hint which JVM works with OEA (including Visual Designer) and supports ibm850?

This thread is closed