Codepage problem with HttpResponse

Posted by Arno van der Ende on 31-Oct-2018 05:57

Is it possible to determine the codepage from a HttpResponse?

I have a response where the Entity is of type 'OpenEdge.Core.Mempt'. However, I expected it to be of type 'OpenEdge.Core.WidgetHandle'. In the logigng I see the Content-Type is changed from application/xml to application/octet-steam:

[18/10/30@14:56:15.191+0100] P-004440 T-003948 1 AS-10 LogMgrWrtr     [OE.N.HTTP.L.ABLS.ABLSocketLibrary ERROR] EXTRACT ENTITY ERROR
[18/10/30@14:56:15.191+0100] P-004440 T-003948 1 AS-10 LogMgrWrtr     	Error(s) raised:
[18/10/30@14:56:15.191+0100] P-004440 T-003948 1 AS-10 LogMgrWrtr     		X-NODEREF or X-DOCUMENT LOAD got an error: FATAL ERROR: file 'MEMPTR', line '1', column '1', message 'exceeded byte limit at byte '�' in a 6-byte sequence'. (9082) (9082)
[18/10/30@14:56:15.191+0100] P-004440 T-003948 1 AS-10 LogMgrWrtr     [OE.N.HTTP.L.ABLS.ABLSocketLibrary ERROR] EXTRACT ENTITY: Response entity contains raw message body
[18/10/30@14:56:15.191+0100] P-004440 T-003948 1 AS-10 LogMgrWrtr     [OE.N.HTTP.L.ABLS.ABLSocketLibrary ERROR] EXTRACT ENTITY: Response Content-Type changed from application/xml to application/octet-stream

I analyzed the request with PostMan, but PostMan also does not show the correct character.

When I read the String from the Memptr, I also get an error:

oMemptrResponseData = CAST(oHttpResponse:Entity, OpenEdge.Core.Memptr).
message oMemptrResponseData:GetString(1).

The GetString error raises error: 

Large object assign or copy failed. (11395)

That error is because of the invalid character in the middle of the response. See also: knowledgebase.progress.com/.../000049503

To get around this, I can read the Memptr response and convert to iso8859-1:

copy-lob from oMemptrResponseData:Value to lcXmlResponseData convert source codepage "iso8859-1".

Now I see the response contains a 'ü' character.

However, how can I know the source is 'iso8859-1'? 

BTW: My session is UTF-8 (CPINTERAL & CPSTREAM).

Some extra code how I get the Response:

CREATE X-DOCUMENT hXmlRequestData.
lcInput = "<?xml version=~"1.0~" encoding=~"UTF-8~"?><BulkLoad><StartRangeTime>2018-10-01</StartRangeTime><EndRangeTime>2018-10-08</EndRangeTime></BulkLoad>".
hXmlRequestData:LOAD("LONGCHAR", lcInput, FALSE).
oXmlRequestData = NEW WidgetHandle(hXmlRequestData).
cUrl = "localhost:18000". // a valid http url
oHttpRequest = RequestBuilder:Post(cUrl, oXmlRequestData):ContentType("application/xml"):AcceptXml():Request.
oHttpResponse = ClientBuilder:Build():Client:Execute(oHttpRequest).

Posted by Matt Baker on 31-Oct-2018 07:12

Code page for an HTTP response is expected to be returned by the server in an HTTP header.  You should be able to fetch the HTTP header from the response.  The header you need is always called "Content-Type".  Should have a value like "application/json; charset=UTF-8" or maybe something like "text/html; charset=ISO8859-1".  The part after the semi-colon will be the code page of the response.

All Replies

Posted by Matt Baker on 31-Oct-2018 07:12

Code page for an HTTP response is expected to be returned by the server in an HTTP header.  You should be able to fetch the HTTP header from the response.  The header you need is always called "Content-Type".  Should have a value like "application/json; charset=UTF-8" or maybe something like "text/html; charset=ISO8859-1".  The part after the semi-colon will be the code page of the response.

Posted by Arno van der Ende on 31-Oct-2018 09:12

Thank you Matt.

In fact the oHttpResponse:ContentType and oHttpResponse:GetHeader("Content-Type") did not contain the charset. However, when I set the charset to 'utf-8' in the request, the response does contain the 'charset=utf-8'. Also the Entity type is of WidgetHandle in that case. So thank you for the tip!

This did the trick:

oHttpRequest = RequestBuilder:Post(cUrl, oXmlRequestData):ContentType("application/xml; charset=utf-8"):AcceptXml():Request.

This thread is closed