utf-8 webclient->appserver->db

Posted by Darren Parr on 26-Apr-2016 08:47

Hi

I'm noticing some strange behaviour. using our application, we notice that russian characters come back to the client after being committed as ?????. We're using cpinternal and cpstream to utf-8 on all clients (appservers,webclient) and the codepage for the log files are set to utf-8 also.

The db is a converted utf-8 db etc. A client/server connection gui connection using utf-8 for cpinternal  to this db works fine with russian text. We can write it and read it and its consistent.

Ive noticed that our application seems to work on the client. ie russian text entered in the webclient gui, displays correctly before its committed. As soon as its written, the encoding seems to be lost. in fact a db trigger shows the question marks in the appserver log file which means the stuff is lost as soon as it goes server side.

This is a legacy app which uses raw-transfer to transfer temp-table data from client to appserver. Before the days of pds, the only way to get multiple temp-tables from client to server using a generic call was this way. Does anyone know if utf-8 data will survive a raw-transfer.

regards

Darren 

All Replies

Posted by Garry Hall on 26-Apr-2016 09:56

RAW-TRANSFER should observe codepage conversion where necessary. From the doc:

"Compares source and target code page ids, and (if they are present and different) translates the source’s character data, writing any warnings to the database log file and raising any error conditions."

You suggest the problem is when going through the appserver. Some thoughts on debugging:

- dump out the raw bytes of the RAW field on each side of the wire (LENGTH, GET-BYTE). That might suggest on which side of the wire to start looking for a problem. Hopefully, your strings will be obvious enough in the data without needing to know the format.

- if not using SSL, you could sniff the traffic, to see if your strings are encoded correctly on the wire.

- there is some undocumented logging which you could turn on to view the network payload, but I'd ask you to log a call with TS for that. It might be easier than dumping out the RAW yourself.

Posted by Darren Parr on 27-Apr-2016 11:37

OK. Thanks. I'll do some more digging. I just wasn't sure if there was anything obvious.

Posted by Garry Hall on 27-Apr-2016 11:49

You covered the obvious when you said the appserver was using -cpinternal UTF-8. That would have be the first thing to check. My gut feel is some unexpected codepage conversion happening on the appserver, but it's worth exploring all avenues.

This thread is closed