JSDO: reduce number of count requests

Posted by Mike Fechner on 06-Aug-2015 18:16

It looks like, the JSDO invokes the "count" method on the AppServer in every case. Even when the prior call to retrieve data returns less rows than requested. 

Isn't it fair to assume, that when the JSDO requests 10 rows and the server returns 0..9 rows, that this is the total number of rows? Or are the actual call to fetch data and the count request executed asynchronously?

Is that something I could add in our JavaScript code? Or would that require a change in the source of the JSDO?

Speaking of possible enhancement requests to the JSDO - would PSC expect us to file an issue on Github for that?

All Replies

Posted by egarcia on 06-Aug-2015 23:03

Hello Mike,

Thank you for the suggestions.

The calls are done in sequence, first read then count.

Returning 0-9 rows when requesting 10 rows means that you are at the last page. However, the number of records may not necessarily be 0-9 since there could be more pages. Each page request could be different from the previous one, for example you could specify a filter. This means that we cannot take advantage (in a simple way) of knowing the previous count.

We could optimize the case for when the first page has been requested and the number of records returned is less than the page size. Is this the case that you are concerned about?

We could also optimize the case for when a filter has not been specified and the last page is returned.

You could add a local count that returns a Promise to override the invoke, however, this change is something that would be better implemented in the JSDO code, specifically in the code that interacts with the Kendo UI DataSource (you can search for countFnName to see where it is used).

In the future we may return the count with a response for read.

Since the JSDO is open source, you could use GitHub or the regular channels for submitting an enhancement request.

Thank you and regards.

Posted by Mike Fechner on 07-Aug-2015 02:22

Hi Edsel,

you are right - of course this is only possible on the first page. But once you navigate from a large overview page many of following grids will typically only return a rather small amount of records - so that this reduction will make sense.

In our GUI for .NET framework we do also on the fly switch from server side sort to client sort, once the client has received all records. This might be an additional optimization (I will test, if that could be set after JSDO data source has received an initial set of data using the data source properties)

Von Outlook gesendet

[collapse]
_____________________________
[collapse] From: egarcia <bounce-egarcia@community.progress.com>
Sent: Freitag, August 7, 2015 06:04
Subject: RE: [Technical Users - Mobile] JSDO: reduce number of count requests
To: <tu.mobile@community.progress.com>


Reply by egarcia

Hello Mike,

Thank you for the suggestions.

The calls are done in sequence, first read then count.

Returning 0-9 rows when requesting 10 rows means that you are at the last page. However, the number of records may not necessarily be 0-9 since there could be more pages. Each page request could be different from the previous one, for example you could specify a filter. This means that we cannot take advantage (in a simple way) of knowing the previous count.

We could optimize the case for when the first page has been requested and the number of records returned is less than the page size. Is this the case that you are concerned about?

We could also optimize the case for when a filter has not been specified and the last page is returned.

You could add a local count that returns a Promise to override the invoke, however, this change is something that would be better implemented in the JSDO code, specifically in the code that interacts with the Kendo UI DataSource (you can search for countFnName to see where it is used).

In the future we may return the count with a response for read.

Since the JSDO is open source, you could use GitHub or the regular channels for submitting an enhancement request.

Thank you and regards.

Stop receiving emails on this subject.

Flag this post as spam/abuse.



[/collapse][/collapse]

Posted by egarcia on 07-Aug-2015 05:45

Hello Mike,

Great. We are on the same page.

I wonder if you would be willing to try some few *unofficial* lines of code in your environment.

I will suggest to include this code in a future release.

If so, here are some few lines of code that does what you are looking for.

Notice the line that tests for options.data.skip. You can find this section by searching for countFnName:

                        if (options.data && options.data.take) {
                            if (!this.readLocal &&
                                transport.countFnName !== undefined &&
                                typeof(jsdo[transport.countFnName]) === "function") {

                                if (options.data.skip === 0 && 
                                    options.data.take > data.data.length) {
                                    options.success(data);
                                    return;
                                }

                                // Reuse filter string from the request.objParam object from fill() call.
                                promise = jsdo.invoke(
                                                transport.countFnName,
                                                { filter: request.objParam.filter });
                                /*jslint unparam: true*/
                                promise.done(function(fnName, jsdo, success, request){
                                    var exception, total;



Please let me know if this works for you.

I hope this helps.

Thanks.

Posted by Mike Fechner on 07-Aug-2015 07:02

Looks promising. Will try that out ASAP!

Posted by Mike Fechner on 09-Aug-2015 08:51

Hi Edsel, seems to do what it should :-)

github.com/.../3b27108718217fbf1dd804ffab00137839addb0b

Posted by egarcia on 09-Aug-2015 17:42

Hello Mike,

Great. Thank you for trying the fix.

I see the pull request with this change and the correction to the header.

We will process the pull request for a future version.

Best regards.

This thread is closed