Problem when running HTTP request from ABL procedure.

Posted by marekk on 06-Nov-2019 14:17

Hi,

I have a problem running the HTTP request for a site with unknown cipher set. The error 9318 appears (file Test_HTTP.p).

However, when I run the request from a WebClient (TestWebClient.p), everything goes fine. Do you have any idea why is that?

Best regards,

Marek


//------------------------------------------------------------------------
//    File        : Test_HTTP.p

/* ***************************  Definitions  ************************** */

USING OpenEdge.Net.HTTP.IHttpRequest.
USING OpenEdge.Net.HTTP.IHttpResponse.
USING OpenEdge.Net.HTTP.ClientBuilder.
USING OpenEdge.Net.HTTP.RequestBuilder.
USING OpenEdge.Net.HTTP.IHttpClientLibrary.
USING OpenEdge.Net.HTTP.Lib.ClientLibraryBuilder.
USING System.IO.* .
USING System.Text.* .
USING progress.Json.*.
USING progress.Json.ObjectModel.* .
/* ***************************  Definitions  ************************** */
/* ********************  Preprocessor Definitions  ******************** */
DO ON ERROR UNDO, THROW:

DEFINE VARIABLE question   AS CHARACTER NO-UNDO.

/* ********************  Preprocessor Definitions  ******************** */


/* ***************************  Main Block  *************************** */
DEFINE VARIABLE oLib AS IHttpClientLibrary NO-UNDO.

DEFINE VARIABLE cSSLProtocols AS CHARACTER EXTENT   NO-UNDO.
DEFINE VARIABLE cSSLCiphers   AS CHARACTER EXTENT   NO-UNDO.
DEFINE VARIABLE oJsonEntity AS JsonObject NO-UNDO.
DEFINE VARIABLE JsonString AS LONGCHAR NO-UNDO.

// the size and values of the SSL protocols and ciphers depend on the server
EXTENT(cSSLProtocols) = 2.
EXTENT(cSSLCiphers) = 10.

// Supported ciphers and protocols at documentation.progress.com/.../supported-protocols,-ciphers,-and-certificates-f.html
ASSIGN cSSLProtocols[1] = 'TLSv1.2'
       cSSLProtocols[2] = 'TLSv1.1'
       cSSLCiphers[1]  = 'AES128-SHA256'
       cSSLCiphers[2]  = 'DHE-RSA-AES128-SHA256'
       cSSLCiphers[3]  = 'AES128-GCM-SHA256'
       cSSLCiphers[4]  = 'DHE-RSA-AES128-GCM-SHA256'
       cSSLCiphers[5]  = 'ADH-AES128-SHA256'
       cSSLCiphers[6]  = 'ADH-AES128-GCM-SHA256'
       cSSLCiphers[7]  = 'ADH-AES256-SHA256'
       cSSLCiphers[8]  = 'AES256-SHA256'
       cSSLCiphers[9]  = 'DHE-RSA-AES256-SHA256'
       cSSLCiphers[10] = 'AES128-SHA'
       

oLib = ClientLibraryBuilder
        :Build()
        :SetSslProtocols(cSSLProtocols)
        :SetSslCiphers(cSSLCiphers)
        :sslVerifyHost(NO)
        :Library.
        
question = "wl-api.mf.gov.pl/.../ .
 
 DEFINE VARIABLE oRequest  AS IHttpRequest NO-UNDO.
DEFINE VARIABLE oResponse AS IHttpResponse NO-UNDO.

oRequest = RequestBuilder:Get(question):Request.
oResponse = ClientBuilder:Build():UsingLibrary(oLib):Client:Execute(oRequest).

oJsonEntity = CAST(oResponse:Entity, JsonObject).
oJsonEntity:Write(JsonString, TRUE).

MESSAGE STRING(JsonString)
VIEW-AS ALERT-BOX.

 
END .
CATCH eAnyError AS Progress.Lang.Error:
  MESSAGE
      "Error Number:" eAnyError:GetMessageNum(1) SKIP
      "Error Text:t" eAnyError:GetMessage(1)
      VIEW-AS ALERT-BOX BUTTONS OK TITLE "Error processing in the CATCH for mainprocedure block".

 RETURN 'OK' .   
END CATCH.

-----------------------------------------------------------------------------------------------------------------------------------------------------------


//------------------------------------------------------------------------
//    File        : TestWebClient.p

/* ***************************  Definitions  ************************** */


/* ********************  Preprocessor Definitions  ******************** */


/* ***************************  Main Block  *************************** */
DEFINE VARIABLE xClient    AS System.Net.WebClient .
DEFINE VARIABLE xWyn       AS LONGCHAR NO-UNDO.
DEFINE VARIABLE question    AS CHARACTER NO-UNDO.

question =  "wl-api.mf.gov.pl/.../ .
 
 
DO ON ERROR UNDO, THROW:
 
System.Net.ServicePointManager:SecurityProtocol = System.Net.SecurityProtocolType:Tls12 .
xClient = NEW System.Net.WebClient () .
xWyn = xClient:DownloadString ( question ) .   


MESSAGE STRING(xWyn)
VIEW-AS ALERT-BOX.

 
END .
CATCH eAnyError AS Progress.Lang.Error:
  MESSAGE
      "Error Number:" eAnyError:GetMessageNum(1) SKIP
      "Error Text:t" eAnyError:GetMessage(1)
      VIEW-AS ALERT-BOX BUTTONS OK TITLE "Error processing in the CATCH for mainprocedure block".

 RETURN 'OK' .   
END CATCH.

Posted by Peter Judge on 07-Nov-2019 17:15

There's a good site at www.ssllabs.com/.../analyze.html that scribes the supported protocols and ciphers for a site.
 
From that,  I can see  that the site supports the following
Cipher Suites
https://www.ssllabs.com/images/collapse.png
# TLS 1.3 (suites in server-preferred order)
TLS_AES_256_GCM_SHA384 (0x1302)   ECDH x25519 (eq. 3072 bits RSA)   FS
256
TLS_CHACHA20_POLY1305_SHA256 (0x1303)   ECDH x25519 (eq. 3072 bits RSA)   FS
256
TLS_AES_128_GCM_SHA256 (0x1301)   ECDH x25519 (eq. 3072 bits RSA)   FS
128
https://www.ssllabs.com/images/collapse.png
# TLS 1.2 (suites in server-preferred order)
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)   ECDH x25519 (eq. 3072 bits RSA)   FS
256
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)   ECDH x25519 (eq. 3072 bits RSA)   FS   WEAK
256
 
 
From that, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 and TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 are supported for TLS1.2 (you're not using TLS1.3).  The OE doc site at docs.progress.com/.../Supported-protocols-ciphers-and-certificates-for-OpenEdge-clients-and-servers.html indicates that the "CBC" cipher is not supported.
 
I tweaked your code to only have
// Supported ciphers and protocols at documentation.progress.com/.../supported-protocols,-ciphers,-and-certificates-f.html
extent(cSSLProtocols) = 1.
extent(cSSLCiphers) = 1.
 
assign cSSLProtocols[1] = 'TLSv1.2'
       cSSLCiphers[1]   = 'TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384'
.
 
 
I found that TLS1.3 didn't work (the socket disconnected).
 
I *also* - and this is where the 9318 message comes from - had to import the site's 3 certificates into the OE cert stort, using certutil -import <cert>.crt
wl-test_mf_gov_pl.crt
DigiCertGlobalRootG2.crt
GeoTrustTLSRSACAG1.crt
 
I got those certificates from my web browser.
 
Once I did all of that, I saw the message below (which I hope you're expecting).
 
---------------------------
Message (Press HELP to view stack trace)
---------------------------
{
  "code": "WL-190",
  "message": "Niepoprawne żądanie."
}
---------------------------
OK   Help  
---------------------------
 
 

All Replies

Posted by Peter Judge on 06-Nov-2019 14:48

In test_http.p you're using the ABL HTTP Client; in testwebclient.p you are using the .NET HTTP client. That's one large difference .
 
What OE version are you using?  Prior to 11.7.3 there was a bug in the way we created the client connection w.r.t SSL siphers and protocols. You can log what the connection parameters are, with a  sufficiently high logging level. Add these lones to your test_http program.
 
log-manager:logfile-name = 'test.log'.
log-manager:logging-level = 5.
log-manager:clear-log().
 
That should write a line with the connect parameters.
 
 

Posted by marekk on 06-Nov-2019 17:00

Thanks, Peter,

I was testing the program in OE 12.1. Below there is a line from test.log:

Connect: -H wl-api.mf.gov.pl -S 443 -ssl -nohostverify  -sslprotocols TLSv1.2,TLSv1.1 -sslciphers AES128-SHA256,DHE-RSA-AES128-SHA256,AES128-GCM-SHA256,DHE-RSA-AES128-GCM-SHA256,ADH-AES128-SHA256,ADH-AES128-GCM-SHA256,ADH-AES256-SHA256,AES256-SHA256,DHE-RSA-AES256-SHA256,AES128-SHA

Marek

Posted by Akshay Guleria on 06-Nov-2019 19:50

You can also set environment var "SSLSYS_DEBUG_LOGGING=5" which will generate SSL layer logs for an ABL client. You can find the log "cert.client.log" in you work dir and it can help you to investigate any potential SSL handshake issues that is creating problems

Posted by Peter Judge on 07-Nov-2019 17:15

There's a good site at www.ssllabs.com/.../analyze.html that scribes the supported protocols and ciphers for a site.
 
From that,  I can see  that the site supports the following
Cipher Suites
https://www.ssllabs.com/images/collapse.png
# TLS 1.3 (suites in server-preferred order)
TLS_AES_256_GCM_SHA384 (0x1302)   ECDH x25519 (eq. 3072 bits RSA)   FS
256
TLS_CHACHA20_POLY1305_SHA256 (0x1303)   ECDH x25519 (eq. 3072 bits RSA)   FS
256
TLS_AES_128_GCM_SHA256 (0x1301)   ECDH x25519 (eq. 3072 bits RSA)   FS
128
https://www.ssllabs.com/images/collapse.png
# TLS 1.2 (suites in server-preferred order)
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)   ECDH x25519 (eq. 3072 bits RSA)   FS
256
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)   ECDH x25519 (eq. 3072 bits RSA)   FS   WEAK
256
 
 
From that, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 and TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 are supported for TLS1.2 (you're not using TLS1.3).  The OE doc site at docs.progress.com/.../Supported-protocols-ciphers-and-certificates-for-OpenEdge-clients-and-servers.html indicates that the "CBC" cipher is not supported.
 
I tweaked your code to only have
// Supported ciphers and protocols at documentation.progress.com/.../supported-protocols,-ciphers,-and-certificates-f.html
extent(cSSLProtocols) = 1.
extent(cSSLCiphers) = 1.
 
assign cSSLProtocols[1] = 'TLSv1.2'
       cSSLCiphers[1]   = 'TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384'
.
 
 
I found that TLS1.3 didn't work (the socket disconnected).
 
I *also* - and this is where the 9318 message comes from - had to import the site's 3 certificates into the OE cert stort, using certutil -import <cert>.crt
wl-test_mf_gov_pl.crt
DigiCertGlobalRootG2.crt
GeoTrustTLSRSACAG1.crt
 
I got those certificates from my web browser.
 
Once I did all of that, I saw the message below (which I hope you're expecting).
 
---------------------------
Message (Press HELP to view stack trace)
---------------------------
{
  "code": "WL-190",
  "message": "Niepoprawne żądanie."
}
---------------------------
OK   Help  
---------------------------
 
 

Posted by marekk on 07-Nov-2019 22:16

Many thanks, Peter for your extraordinary help!

Marek

This thread is closed