Bug with rbv_api.sendHttpPost(url, params, headers)

Posted by surferking on 14-Nov-2013 11:08

rbv_api.sendHttpPost(url, params, headers)

when headers = null, it properly posts data to url, if headers are set, it unsets the data in the request (data = null).

most probably it's a bug

All Replies

Posted by Laurent on 14-Nov-2013 13:46

I did a test with a random header and it seems to work. Could you give me the exact headers you used?

Posted by surferking on 15-Nov-2013 01:41

Here is the trigger code:

[code]

// base64_encode (client_id:secret)

var basic_auth_token = "RU9KMlMtWjZPb05fbGVfS1MxZDc1d3NaNnkwU0ZkVnNZOTE4M0l2eEZ5WnA6RUNsdXNNRVVrOGU5aWhJN1pkVkxGNWNaNnkwU0ZkVnNZOTE4M0l2eEZ5WnA=";

var headers = {
               "Accept": "application/json",
               "Accept-Language": "en_US",
               "Authorization": "Basic " + basic_auth_token
   };
rbv_api.println("Headers:");
rbv_api.printArr(headers);
rbv_api.println("");

rbv_api.println("Data:");
var data = {"grant_type": "client_credentials"};
rbv_api.printArr(data);
rbv_api.println("");

var url = "http://mysite.com/test.php";


var resp = rbv_api.sendHttpPost(url, data, headers);

rbv_api.println("Response:");
rbv_api.printArr(resp);
rbv_api.println("");

[/code]

test.php code:

[code]

<?php

print_r($_SERVER);

print_r($_POST);

?>

[/code]

If we send the above headers, we get empty _POST array in the php script.

If we send headers=null, we get _POST array properly.

Posted by surferking on 19-Nov-2013 06:31

Any news on this please? We need this bug fixed so that we can complete our creation of a Paypal payment Object using their REST API

Posted by Laurent on 26-Nov-2013 17:55

Apologies, for some reason I wasn't notified of your response (or missed the notification).

Either way, is the PHP code only the two lines you're showing? When I try to use your code I get "Error  Upload failed, response=Not Acceptable" (whether the headers passed are null or not)

Posted by surferking on 27-Nov-2013 06:51

Have you changed the URL to your appropriate test.php file location?

var url = "mysite.com/test.php";

Posted by Laurent on 27-Nov-2013 12:44

Yes I did. I made sure that the test.php file had the exact same content as you.

Posted by surferking on 28-Nov-2013 04:16

test.php content can be:

<?php

print "<h1>POSTED DATA:</h1><br /><pre>";

print_r($_POST);

print "</pre>";

print "<h1>SERVER VARS (headers and others)</h1><br /><pre>";

print_r($_SERVER);

print "</pre>";

?>

After that, change url in the trigger javascript to the actual url of test.php on your server.

Please let me know if that helps you re-produce the issue. Also, what version of RB are you testing on?

Posted by surferking on 28-Nov-2013 08:09

We seem to be having the same issue on RB v2.0.4 so it seems it is still an outstanding bug

Posted by Laurent on 29-Nov-2013 08:48

I tried with both Rollbase Hosted Cloud (so 2.0.0.4) and Private Cloud 2.0.0.3. And I doubt that the second version of the PHP code changes anything, as all it does is to print extra HTML. The error I get seems to be HTTP error 406, which means I'll try to pass some headers typically passed by a web browser.

Posted by Laurent on 29-Nov-2013 12:39

By the way, what trigger type are you using to run the JavaScript code? The type shouldn't matter, but one never knows.

Posted by Laurent on 29-Nov-2013 13:11

Here is a test that I performed and that seems to work even with headers:

var url = "hroch486.icpf.cas.cz/.../echo.pl";

var params = { "your_name": "This is a test" }

var headers = { "Accept": "text/html" }

var resp = rbv_api.sendHttpPost(url, params, headers);

I created an Object Script type of trigger and tested the code using the Debug Formula. Whether I use headers or not (or even your headers) I get a response.

Don't get me wrong, I am not saying there is no bug in Rollbase. But if there is, it seems to be more than just the lack of headers.

Posted by surferking on 02-Dec-2013 06:55

We use Object Script type of triggers and test it using the Debug Formula.

We have just realized that  sendHttpPost() sends "Content-Type: multipart/form-data" header.

Is it possible to change it to "Content-Type: application/x-www-form-urlencoded"?

Our aim is to implement PayPal Rest API, for example:

[quote]

curl -v https://api.sandbox.paypal.com/v1/oauth2/token -H "Accept: application/json"  -H "Accept-Language: en_US"   -u "EOJ2S-Z6OoN_le_KS1d75wsZ6y0SFdVsY9183IvxFyZp:EClusMEUk8e9ihI7ZdVLF5cZ6y0SFdVsY9183IvxFyZp"  -d "grant_type=client_credentials"

[/quote]

If we send "Content-Type: multipart/form-data" via curl to PayPal, we get "415 Unsupported Media Type" error i.e. the same error in Rollbase.

[quote]

curl -v https://api.sandbox.paypal.com/v1/oauth2/token  -H "Accept: application/json"  -H "Accept-Language: en_US"  -H "Content-Type: multipart/form-data"  -u "EOJ2S-Z6OoN_le_KS1d75wsZ6y0SFdVsY9183IvxFyZp:EClusMEUk8e9ihI7ZdVLF5cZ6y0SFdVsY9183IvxFyZp"  -d "grant_type=client_credentials"

[/quote]

We suspect Rollbase doesn't take into account "Content-Type" header and incorrectly performs the request i.e. it sends data as multipart anyway.

[quote]

var headers = {
               "Accept": "application/json",
               "Accept-Language": "en_US",
               "Content-Type": "application/x-www-form-urlencoded",
               "Authorization": "Basic " + basic_auth_token
   };

[/quote]

Posted by Anoop Premachandran on 02-Dec-2013 07:31

You can use trigger of type "Send HTTP POST Request" and here select Encoding as URL to set Content Type as application/x-www-form-urlencoded

Posted by surferking on 03-Dec-2013 01:44

It seems there is another issue with "Send HTTP POST Request" type of trigger, it doesn't fill in the "Content-Length" header in the request.

[quote]

<title>411 Length Required</title>

</head><body>

<h1>Length Required</h1>

<p>A request of the requested method POST requires a valid Content-length.<br />

...

[/quote]

Anyway we need to a logic to perform several POST requests so we think  "Send HTTP POST Request" type of trigger won't work for us.

Posted by murali_pacific on 04-Dec-2013 05:31

Hello,

As you have already discovered, the sendHttpPost  

api sends it as multi part  request, and uses chunked encoding.

I have tested your trigger code with headers and without headers on a developer build.  

I also put in a customer header in the request.

And with the same php ( running on wamp 2.2).  It does show the correct POST data on all tests.

So a couple of questions for you,

1. What version of RB are you having trouble with?

2. What is the environment on which you host php? ( Not that this is where the likely problem is.. but just eliminating any suspicion if I can)

Posted by surferking on 04-Dec-2013 06:14

> 1. What version of RB are you having trouble with?

1.0.0.0 and 2.0.0.3

> 2. What is the environment on which you host php?

Apache/2.2.3 (CentOS), PHP version 5.3.3

When php receives the posted data there is only $HTTP_RAW_POST_DATA array populated, $_POST array is empty.

sample of $HTTP_RAW_POST_DATA array:

--3fvU0jhZR-TIk2pNNMcEabbbFwd2iAw5q

Content-Disposition: form-data; name="grant_type"

Content-Type: text/plain; charset=UTF-8

Content-Transfer-Encoding: 8bit

client_credentials

--3fvU0jhZR-TIk2pNNMcEabbbFwd2iAw5q--

PHP code:

<?php

print_r($_SERVER);

print_r($HTTP_RAW_POST_DATA);

print_r($_POST);

?>

Posted by surferking on 04-Dec-2013 06:15

Arrrrggghhh! This comments box still doesn't accept Code

Posted by surferking on 04-Dec-2013 06:16

> 1. What version of RB are you having trouble with?

1.0.0.0 and 2.0.0.3

> 2. What is the environment on which you host php?

Apache/2.2.3 (CentOS), PHP version 5.3.3

When php receives the posted data there is only $HTTP_RAW_POST_DATA array populated, $_POST array is empty.

sample of $HTTP_RAW_POST_DATA array:

/*

--3fvU0jhZR-TIk2pNNMcEabbbFwd2iAw5q

Content-Disposition: form-data; name="grant_type"

Content-Type: text/plain; charset=UTF-8

Content-Transfer-Encoding: 8bit

client_credentials

--3fvU0jhZR-TIk2pNNMcEabbbFwd2iAw5q--

PHP code:

<?php

print_r($_SERVER);

print_r($HTTP_RAW_POST_DATA);

print_r($_POST);

?>

*/

Posted by murali_pacific on 04-Dec-2013 10:15

Hello, As you have already discovered, the sendHttpPost api sends it as multi part request, and uses chunked encoding. I have tested your trigger code with headers and without headers on a developer build. I also put in a customer header in the request. And with the same php ( running on wamp 2.2). It does show the correct POST data on all tests.

So a couple of questions for you,

1. What version of RB are you having trouble with?

2. What is the environment on which you host php?

Posted by murali_pacific on 04-Dec-2013 10:16

OK. Let me test on 2.0.0.3 and get back to you.

Posted by surferking on 04-Dec-2013 10:17

Thanks for helping. I already answered these questions above...

Posted by surferking on 04-Dec-2013 10:22

Same issue on 2.0.0.4

Posted by murali_pacific on 04-Dec-2013 10:55

Hi, I'm trying to check if your php supports  multipart/form-data.

Can you have an html which contains the below form & submit it and see if you get  the form values  $_POST ? 

<form action="your_target.php"
method="post" enctype="multipart/form-data">
First name: <input type="text" name="fname"><br>
Last name: <input type="text" name="lname"><br>
<input type="submit" value="Submit">

</form>

Your previous post (raw post data) shows that the data is reaching the server. 

--3fvU0jhZR-TIk2pNNMcEabbbFwd2iAw5q

Content-Disposition: form-data; name="grant_type"

Content-Type: text/plain; charset=UTF-8

Content-Transfer-Encoding: 8bit

client_credentials

--3fvU0jhZR-TIk2pNNMcEabbbFwd2iAw5q--

Another thought ( need to verify ) : This may be our issue https://bugs.php.net/bug.php?id=50210

Posted by surferking on 05-Dec-2013 02:23

Here is our php script test.php:

[quote]

<?php

if ($_SERVER["REQUEST_METHOD"] == "POST") {

       print "<h1>Server</h1>\n<pre>";

       print_r($_SERVER);

       print "</pre><h1>REQUEST</h1>\n<pre>";

       print_r($_REQUEST);

       print "</pre><h1>RAW POST DATA</h1>\n<pre>";

       if (isset($HTTP_RAW_POST_DATA) && !empty($HTTP_RAW_POST_DATA))

               print_r($HTTP_RAW_POST_DATA);

       else

               print "empty...\n\n";

       print "</pre><h1>POST</h1>\n<pre>";

       print_r($_POST);

       print "</pre>";

}

?>

<form action="our_script.php"

method="post" enctype="multipart/form-data">

First name: <input type="text" name="fname"><br>

Last name: <input type="text" name="lname"><br>

<input type="submit" value="Submit">

</form>

[/quote]

Output:

[quote]

Server

Array
(
    [SCRIPT_URL] => /test.php
    [SCRIPT_URI] => http://oursite.com/test.php
    [HTTP_HOST] => oursite.com
    [HTTP_USER_AGENT] => Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20131101 Firefox/24.0 Iceweasel/24.1.0
    [HTTP_ACCEPT] => text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    [HTTP_ACCEPT_LANGUAGE] => en-US,en;q=0.5
    [HTTP_ACCEPT_ENCODING] => gzip, deflate
    [HTTP_REFERER] => http://oursite.com/test.php
    [CONTENT_TYPE] => multipart/form-data; boundary=---------------------------3621816720535931238390484
    [CONTENT_LENGTH] => 282
    [HTTP_CACHE_CONTROL] => max-age=259200
    [HTTP_CONNECTION] => keep-alive
    [PATH] => /usr/local/bin:/usr/bin:/bin
    [SERVER_SIGNATURE] =>
Apache/2.2.9 (Debian) mod_gnutls/0.5.1 PHP/5.2.6-1+lenny16 with Suhosin-Patch mod_ssl/2.2.9 OpenSSL/0.9.8g mod_wsgi/2.5 Python/2.5.2 Server at oursite.com Port 80
    [SERVER_SOFTWARE] => Apache/2.2.9 (Debian) mod_gnutls/0.5.1 PHP/5.2.6-1+lenny16 with Suhosin-Patch mod_ssl/2.2.9 OpenSSL/0.9.8g mod_wsgi/2.5 Python/2.5.2
    [SERVER_NAME] => oursite.com
    [SERVER_ADDR] => 192.168.20.108
    [SERVER_PORT] => 80
    [REMOTE_ADDR] => 192.168.20.108
    [DOCUMENT_ROOT] => /home/vhosts/oursite/httpdocs
    [SERVER_ADMIN] => [no address given]
    [SCRIPT_FILENAME] => /home/vhosts/oursite/httpdocs/test.php
    [REMOTE_PORT] => 39385
    [GATEWAY_INTERFACE] => CGI/1.1
    [SERVER_PROTOCOL] => HTTP/1.1
    [REQUEST_METHOD] => POST
    [QUERY_STRING] => 
    [REQUEST_URI] => /test.php
    [SCRIPT_NAME] => /test.php
    [PHP_SELF] => /test.php
    [REQUEST_TIME] => 1386229137
    [argv] => Array
        (
        )

    [argc] => 0
)

REQUEST

Array
(
    [fname] => Tester
    [lname] => Tester
)

RAW POST DATA

empty...

POST

Array
(
    [fname] => Tester
    [lname] => Tester
)

[/quote]

As for your thought about the php bug, it may be related to but the php script is just a way to debug requests sent by RollBase as actually we need to implement PayPal REST API and PayPal doesn't also accept POST from RollBase.

For example,  we implement Authentication request (see below) in RollBase and we get  "415 Unsupported Media Type" error. 

So we think it's not only PHP issue with a data submitted by RollBase.

[quote]

curl -v https://api.sandbox.paypal.com/v1/oauth2/token -H "Accept: application/json"  -H "Accept-Language: en_US"   -u "EOJ2S-Z6OoN_le_KS1d75wsZ6y0SFdVsY9183IvxFyZp:EClusMEUk8e9ihI7ZdVLF5cZ6y0SFdVsY9183IvxFyZp"  -d "grant_type=client_credentials"

[/quote]

See https://developer.paypal.com/webapps/developer/docs/api/ for a reference.

Posted by Ranjana sinha on 10-Dec-2013 04:00

This is an issue and has been logged as a defect. We will be fixing it in the future releases.

Posted by surferking on 10-Dec-2013 04:36

We are a business that has invested in the Rollbase system to use for daily operations.

"We will be fixing it in the future releases" is not an acceptable answer.

I would like a timescale please? And for updates if that timescale cannot be met.

This is a bug, with a well-articulated and reproduceable description, for a feature that is not working as advertised, that has taken amonth just to get acknowledged - I would like a date when this will be fixed.

This Private Cloud is currently not very impressed with the Progress.com development speed, and understanding of Rollbase development needs - I would like to see more urgent effort to remedy this.

Currently it appears that Rollbase development is almost stagnant - which is not encouraging for investing in a stystem that people will rely upon for daily operations.

This thread is closed