OS-COMMAND Escape Ampersand Inconsistency

Posted by joshreimer on 11-Sep-2019 19:30

I'm trying to do a GET using Curl via OS-COMMAND.  When my program runs on an appserver, I have to escape my ampersands using ^.  This is documented here:

https://knowledgebase.progress.com/articles/Article/000029461  But when my program runs on a web broker, the ^ is not recognized as an escape character and it makes the URL incorrect.

Why are they different?  Can I count on appservers to always need the escape character and web brokers to always not need it?  This will be deployed on many different servers and it's not possible to test each one.

Here is what I am passing to OS-COMMAND:

\\pathtocurl\curl.exe  -X GET -H "Content-Type: application/json" -H "Authorization: Bearer tokenabc123" -s -i -v  --stderr "C:\logfile.err" -o "C:\data.rec" "url.com/endpoint

Or (with the escape character):

\\pathtocurl\curl.exe  -X GET -H "Content-Type: application/json" -H "Authorization: Bearer tokenabc123" -s -i -v  --stderr "C:\logfile.err" -o "C:\data.rec" "url.com/endpoint

Thank you!

All Replies

Posted by Peter Judge on 11-Sep-2019 21:00

This won't solve your OS-COMMAND issue but as of 11.5-ish there's an ABL HTTP client , that's intended for these kinds of requests.
See docs.progress.com/.../Introduction.html for info. You'll have to add the netlib/OpenEdge.Net.pl procedure library to PROPATH.

Posted by ChUIMonster on 11-Sep-2019 21:37

I don't see any ampersands or carets in your example.  What am I missing?

The kbase isn't very clear about it but, as I read it, the problem is that you need an os-appropriate escape character in the string passed to the operating system that is executing the command.  Different operating systems use different escape characters.  ^ is an escape character for Windows CMD shells.  It does not function as an escape for any UNIX shell.  And it does not function as an escape for 4gl code either.

I think what is probably happening is that your app servers are running on Windows and your web brokers are running UNIX.

If that is correct then you also have the problem of "\" being the wrong path delimiter for UNIX and ".exe".

Something like this should work for you:

if opsys = "windows" then

 os-command value( "your windows command with ^ as an escape" ).


 os-command value( "your unix command" ).

In 4gl code "~" is the most appropriate escape character.  "\" is problematic because windows code thinks it is a path delimiter but UNIX code thinks it is an escape.

Posted by joshreimer on 12-Sep-2019 10:43

(This was a reply to Peter Judge.  I'm not sure why it isn't showing up under his post.  Sorry, I'm new here!)

I looked into that and it might be something I do eventually, but it will be a big undertaking to update the propath for all our customers.

Thank you!

Posted by joshreimer on 12-Sep-2019 11:13

Sorry, it looks like my examples both got cut off at the question mark part of the url.  The forum doesn't seem to let me preview my post -- it just disappears until a moderator approves it.

Both the web broker and appserver are on windows (though that won't necessarily be the case in every deployed environment).  OPSYS returns "WIN32".  I tested web brokers on two different machines (both WIN32) and neither one works with the escape character.

The tilde escape character doesn't work because the code is running in the cmd shell, not in 4gl.  A single tilde doesn't show up in the string that gets passed to the shell so it doesn't change anything.  With a double tilde, one of them gets through to the shell and is included in the URL, which then gives me the wrong data.

Here is my example again.  I'll spell out the question mark and ampersand this time and hopefully nothing will get cut off.  The example with the caret would be the same except with a caret before the ampersand.

\\pathtocurl\curl.exe -X GET -H "Content-Type: application/json" -H "Authorization: Bearer mytoken" -s -i -v --stderr "C:\log.err" -o "C:\data.rec" "https://api.com/endpoint [questionmark] param1=abc [amp] param2=123"

Thank you!

Posted by David Abdala on 12-Sep-2019 11:30

I've had similar issues, specially when dealing with Unix and Windows.

The simple solution is to make a bat script (sh on linux) and invoke that script.

If you need to pass parameters to the script from ABL, use an intermediate file, where each line is a parameter (or give it an ini format, or json, or xml, or...).

No need to scape anything in ABL, if scaping is needed,  implement it in the bat/sh script.

Posted by ChUIMonster on 12-Sep-2019 18:47

I mentioned the tilde as the neutral 4gl string escape character because of the \ in your path names.  You are absolutely correct -- it is irrelevant in reference to escaping things once the string is passed to the OS.  But if you do ever need to operate in a mixed environment tilde will serve you well for strings at the 4gl level.

The brain-twisting part of this is that there are multiple layers -- there are some things that have to be escaped at the 4gl level and then others which need to be escaped at the opsys level.  

Where is your code that tests OPSYS?  Are you testing it a runtime?  Or using the pre-processor?

The suggestion to create a BAT file to hide the details can be pretty helpful.

This thread is closed