Slow connection between AIA and appserver

You can ignore the 'ASK' message in the broker log.   It is simply a message indicating the optional keep-alive feature is not enabled.

If your CONNECT -URL has 'HTTP' or 'HTTPS' in it - you will be going through a [Tomcat] web server and AIA.

If your CONNECT parameters hasT -H and -A, then your client connects directly to the AppServer using NameServer service resolution

You indicate that a browser to AIA (via Tomcat) , and a client to AppServer is very quick.   But a client to AppServer is very slow.   The difference would be the connection from the AIA to the AppServer, so it appears that would  be your problem area to focus on.

Your -URL connection parameter indicates that AIA will use the NameServer to locate AS_DP18001.   You can optionally configure the AIA to do a direct connection to the AppServer without the use of a NameServer.    The direct connections generally work very fast as there is no delay in AIA contacting a NameServer and waiting for a response.  It is something that you could consider trying to determine if the NameServer resolution by AIA is causing your slow connection problem.

Hi,

Thanks for the response.

How can I connect from the AIA directly to the Appserver in the -URL without the NS ?

Now we will have two clients, so two databases and two appservers. In the near future we will implement Multi-tenant, but we first want to set this up.

I have exactly the same on my windows 7 PC (OE11.3.2 - Tomcat 7) and that works fast. The same configuration on the Windows 2012R2 server is slow. Can this be caused by windows 2012 or a configuration of the people in the datacenter ?

Kind regards,

Bart Syryn

It would appear the slowness could be something to do with the NameServer. either the use of UDP, or more likely, a DNS lookup. To specify a direct connect with AIA, remove the value for controllingNameServer, and make sure Host and Port are set properly to your appserver.

When I connect directly from AIA to Appserver, the connection is setup fast, so it seems that the NS is the bottleneck.

But go directly to the appserver isn't an option because we have different AppServer/DB's. We're planning to set up multitenancy in a couple of months, that would solve the problem, but it's not possible at this moment....

I would first check if NameServer's java process uses ipv4 or ipv6 protocol, followed by checking the DNS (eventually entries for localhost in C:\Windows\System32\drivers\etc\hosts file) and finally firewall rules.

Use standard Windows "netstat -ab" to double check (search for NS listening UDP port - probably 5162).

Check from proenv> nsman -i NS1 -query

and see how are AppServer broker registered with the NameServer - observe host name/ip which will reveal localhost/ip/hostname being used.

If everything is running on the same machine AND

broker registers with 'localhost' with the NameServer AND

you have 'localhost' lines for ipv4 and ipv6 in Windows hosts file, THEN

you MAY be ok. :)

hi,

Thanks for the reply.

Can you help me how I can check if java process uses ipv4 or ipv6 protocol ?

There's nothing defined in the hosts-file.

Checked 'proenv > nsman -i NS1 -query' and everything seems ok.  The NS connects not with localhost, but the ip-adress of the server. But if I just tried to connect to AIA and appserver with localhost or the internal IP-adres or with 127.0.0.1, than the result is always the same, between 10 and 15sec to connect to the appserver.

>netstat -ab : i don't see any entry on port 5162....

I really don't known what could be the issue here.

Kind regards,

Bart Syryn

netstat -ab takes a while to process

 UDP    0.0.0.0:5162           *:*                    

[java.exe]

If you don't see it then your NS is either not running, or running on a different port.

Why don't you use the wireshark to sniff the network when doing the whatever is slow? You will see where the delay exactly lies. Also you haven't answered the question - how is the broker registered on the Nameserver (pasting nsman -query here would answer that).

Hi,

netstat -ab gives me the following two results  :

1. UDP [::]:5162 {java.exe]

2. UDP [0.0.0.0]:5162 {java.exe]

proenv> nsman -i NS1 -query gives the following result :

NS1 running on Host app001 port 5162 Timeout 30 seconds

AS.AS_DPL18001 numbers:numbers:numbers:numbers app001/ipadresserver port 3091

AS.AS_DPL18002 numbers:numbers:numbers:numbers app001/ipadresserver port 3092

So I don't know if this is correct ....

If what you say is correct and that is that direct connect from AIA to the AppServer broker has no delay, then you need to look at the NameServer round trip.

The way how it works is this:

1. On incoming request AIA sends UDP packet to the host specified in the controlingNameServer property for the specific AIA instance : in your case probably AIA1

2. AIA (SRC some dynamic UDP port) ----> UDP (DST 5162) -----> NameServer  (java.exe listening in your case both on ipv4 and ipv6). This packet will contain value of the AppService that is being requested, say AS_DPL18001)

3. This step will be repeated until NameServer replies back until timeout kicks in (you will see that in wireshark)

4. NameServer looks into the list of registered brokers and for AS_DPL18001 finds the entry and it sends UDP packet from SRC 5162 to the AIA thread listening on originating port from the previous step. That packet will contain app001/ipadresserver as well as the port number where the AppServer broker listens.

5. AIA now switches to TCP and performs the DNS lookup for the host returned by the NameServer

6. Once that is resolved, AIA thread makes TCP connection to the AppServer broker and from that point on everything is the same as with the broker direct connection.

So you have several ways to check network issue here: use wireshark, use ping app001 or ping 4 app001, check DNS reply with nslookup app001, check C:\Windows\System32\drivers\etc\hosts and try adding line:

127.0.0.1    app001

etc.

Hi,

Thanks for the explanation. It's clear now.

Tried ping and nslookup, that all seems ok (ping reply <1ms and nslookup directly).

I changed the host file, but that didn't help.

Wireshark is installed and I did a logging. But with my knowledge of network etc, I don't understand what is in the logging. I don't know where to look or for what in the logging...

Kind regards

Bart Syryn

Hi Bart,

The only way forward is to analyze the Wireshark captured network trace log file. If you can provide remote access to that  machine, then we could schedule some time this week and I can give it a try. Let me know.

-Sasha

Hi Sasha,

It would be a very great help if you would like to give it a try.

About 10 days ago I also placed a support call to progress, but I haven't heard from them the last week....

I really don't know where to look in the wireshark logging.

I suggest that you can take over my computer with teamviewer. From there I have a connection to the server in the datacenter (Remote Desktop).

I don't know in which timezone you are. I'm from Belgium - Brussel.

I have some meetings this week, but I'm available wednesday morning, thursday and Friday.

Maybe you can mail me directly to see when it suits you best to look at the loggings (bart.syryn@advani.be)

Kind regards,

Bart Syryn

To wrap up this topic, after checking with Bart, the issue was a delay introduced by DNS trying to resolve the IP address of the AppServer host, as it was using registrationMode=Register-IP. This has been tested by adding in the Windows hosts file line:

xxx.xxx.xxx.xxx      xxx.xxx.xxx.xxx

where 'xxx.xxx.xxx.xxx' represents the ip address with which AppServer registered with the NameServer.

Suggestion is to use following in the AppServer broker configuration (ubroker.properties file) :

registrationMode=Register-HostName

hostName=<your-host-name>

where <your-host-name> is being properly resolved by the DNS. This is especially important if your AppServer broker runs on the machine behind NAted firewall.

Hi Sasha,

Indeed that solved the problem.

Thanks for all the replies and the excellent solution.

Kind regards

Bart Syryn