Codepage headache

Posted by Patrick Tingen on 06-Jul-2016 07:51

My experiences with codepages is that most of the time they only cause severe headache. Unfortunately, I have yet another case at hand, but perhaps some wise people here can help me clarify some things. 

I have a UTF-8 database and start it using a UTF-8 session (all 11.6). All -cpXX are set to UTF-8 where appropriate:

-cpstream   UTF-8
-cpcase     BASIC
-cpinternal UTF-8
-cplog      UTF-8
-cpterm     UTF-8
-cpcoll     BASIC

My Windows uses 1252. Now, when I try to enter the Hungarian letter "Š", it starts falling apart. I am not able to enter the character. If I try it looks as if I am entering a space:

But when I press F2 it all shows fine:

Both codepages contain the Hungarian S, so this should not be a problem, right? 

Posted by Garry Hall on 06-Jul-2016 08:39

Without digging into it too deeply, I suspect a font issue. I can see this char if I change fonts (Font2), but I have not found a fixed width font that supports it yet. Although it works with -cpinternal 125 and the default font. Seems like a problem with the Procedure Editor's editor control and UTF-8. Other non-ASCII chars are fine ("å"). You could contact TS to log a bug for it.

How often do you plan to use non-ASCII characters in your actual code? This is a simple test to demonstrate a problem in the Procedure Editor, but how often do you really put those characters into code, that you would be inconvenienced by this? Non-ASCII chars in code require all compile environments to use the same -cpstream, otherwise your strings might be interpreted incorrectly.

All Replies

Posted by Garry Hall on 06-Jul-2016 08:39

Without digging into it too deeply, I suspect a font issue. I can see this char if I change fonts (Font2), but I have not found a fixed width font that supports it yet. Although it works with -cpinternal 125 and the default font. Seems like a problem with the Procedure Editor's editor control and UTF-8. Other non-ASCII chars are fine ("å"). You could contact TS to log a bug for it.

How often do you plan to use non-ASCII characters in your actual code? This is a simple test to demonstrate a problem in the Procedure Editor, but how often do you really put those characters into code, that you would be inconvenienced by this? Non-ASCII chars in code require all compile environments to use the same -cpstream, otherwise your strings might be interpreted incorrectly.

Posted by Aidan Jeffery on 06-Jul-2016 08:50

The database code page doesn’t seem to be relevant to the problem you are seeing. It will depend on the client startup parameters and the fonts that are used - in this case, the font that is used for the widget that you are entering the character value into.
Assuming that your installation is using Courier New for font2, try this:
 
DEFINE VARIABLE myChar AS CHARACTER INITIAL "Š".
 
UPDATE myChar WITH 1 COLUMN FONT 2.
 
MESSAGE "myChar =" myChar
  VIEW-AS ALERT-BOX.
 
 

Posted by Patrick Tingen on 06-Jul-2016 08:53

I am strongly against using "strange" characters in source code. The example above was just to show that it is impossible to enter the character, although it can be viewed. I should have added a screenshot with the result of a display-statement. If you use a display, the character can't be show either.

After experimenting with the font, I must conclude that you are right; is /is/ a font issue. I selected another font and I was able to show the character just fine.

Posted by Garry Hall on 06-Jul-2016 09:07

The problems with character rendering in the Procedure Editor and in the DISPLAY statement are different issues, although they both appear to be font related.

display "Š".

with the default font gives me a partial square, a character that cannot be rendered. If I change this to:

display "Š" with font 2.

I can see the character. The same thing happened for the UPDATE statement, I was able to enter (and see) the character when using font 2.

Posted by Torben on 11-Jul-2016 04:16

I've tested on 64 and 32 bit 11.6.2.

On 64 bit the "Š" display correct in editor.

On 32 bit the "Š" display as " ".

So font related for font2 in editor

/Torben

Posted by Brian K. Maher on 11-Jul-2016 04:32

Torben,
 
Dump out the value of session:cpinternal, session:cpterm and session:cpstream for both versions and compare.
 
Brian

Posted by Torben on 19-Jul-2016 07:27

On both 32 and 64 bit the following statement:


Message "Š" SKIP
session:cpinternal SKIP
session:cpterm SKIP
session:cpstream
view-as alert-box.

Show as:

---------------------------
Message
---------------------------
Š
UTF-8
UTF-8
UTF-8
---------------------------
OK
---------------------------

32 bit editor screen looks like:

64 bit editor screen looks like:

This thread is closed