Hello,
OS:SUSE Linux Enterprise Server 11 SP1
OpenEdge: 11.5.01.025
The Customer has just been updated to OpenEdge 11.5.1.025 from 10.2B.
After starting AdminServer using high amounts of CPU until it is stopped. AdminServer not have any resources for autostart (no AppServer & WebSpeed brokers & databases)
In the SAR, we see that immediately after the start of the system CPU value dramatically increases and it remains high even after the start.
08:34:27 AM CPU %user %nice %system %iowait %steal %idle 08:34:32 AM all 6.91 0.00 2.86 4.43 0.00 85.80 08:34:37 AM all 5.99 0.00 2.54 6.14 0.00 85.32 08:34:42 AM all 6.03 0.00 2.47 7.70 0.00 83.80 08:34:47 AM all 5.80 0.00 2.09 7.89 0.00 84.22 08:34:52 AM all 10.27 0.00 2.39 8.27 0.00 79.06 08:34:57 AM all 6.19 0.00 2.20 8.76 0.00 82.86 08:35:02 AM all 7.20 0.00 3.30 14.57 0.00 74.93 ... ... ... 08:41:27 AM all 17.87 0.00 7.05 13.24 0.00 61.85 ------------ AdminServer start point 08:41:32 AM all 16.64 0.00 9.02 12.19 0.00 62.15 08:41:37 AM all 11.43 0.00 8.75 14.59 0.00 65.23 08:41:42 AM all 13.44 0.00 8.73 14.57 0.00 63.25 08:41:47 AM all 12.37 0.00 7.92 13.60 0.00 66.12 08:41:52 AM all 11.05 0.00 8.05 14.87 0.00 66.03 08:41:57 AM all 13.34 0.00 8.32 15.13 0.00 63.21 08:42:02 AM all 12.77 0.00 12.28 9.16 0.00 65.79 08:42:07 AM all 13.43 0.00 14.20 8.37 0.00 64.00 08:42:12 AM all 8.95 0.00 12.18 4.97 0.00 73.91 08:42:17 AM all 8.50 0.00 14.91 4.61 0.00 71.99 08:42:22 AM all 9.36 0.00 15.55 6.58 0.00 68.52 08:42:27 AM all 11.40 0.00 15.65 6.69 0.00 66.26 08:42:32 AM all 10.38 0.00 15.97 7.84 0.00 65.81 08:42:37 AM all 14.84 0.00 13.73 8.96 0.00 62.46 08:42:42 AM all 20.81 0.00 23.79 9.53 0.00 45.87 08:42:47 AM all 25.90 0.00 52.66 7.96 0.00 13.48 08:42:52 AM all 28.95 0.00 58.61 3.43 0.00 9.01 08:42:57 AM all 29.98 0.00 53.98 5.53 0.00 10.50 08:43:02 AM all 28.94 0.00 68.42 0.45 0.00 2.19 08:43:07 AM all 29.31 0.00 69.65 0.06 0.00 0.99 --- AdminServer start end point (Fathom probe started.) 08:43:12 AM all 30.39 0.00 69.20 0.05 0.00 0.36 08:43:17 AM all 30.20 0.00 68.91 0.40 0.00 0.49 08:43:22 AM all 30.80 0.00 65.05 1.22 0.00 2.93 08:43:27 AM all 28.88 0.00 63.89 2.51 0.00 4.72 08:43:32 AM all 28.52 0.00 67.04 1.23 0.00 3.21 08:43:37 AM all 30.83 0.00 64.70 1.23 0.00 3.24 08:43:42 AM all 28.32 0.00 69.24 1.11 0.00 1.33 08:43:47 AM all 25.85 0.00 68.26 1.43 0.00 4.46 08:43:52 AM all 26.10 0.00 67.16 0.72 0.00 6.02 08:43:57 AM all 26.10 0.00 66.30 0.91 0.00 6.69 08:44:02 AM all 26.86 0.00 68.25 1.22 0.00 3.67 08:44:07 AM all 28.23 0.00 65.26 1.05 0.00 5.46 08:44:12 AM all 24.97 0.00 71.88 0.68 0.00 2.47 08:44:17 AM all 25.29 0.00 66.57 1.16 0.00 6.98 08:44:22 AM all 23.32 0.00 68.89 2.18 0.00 5.61 08:44:27 AM all 21.37 0.00 67.84 2.56 0.00 8.22 08:44:32 AM all 23.29 0.00 70.27 1.44 0.00 5.01 08:44:37 AM all 24.21 0.00 68.44 1.03 0.00 6.32 08:44:42 AM all 18.67 0.00 70.15 2.44 0.00 8.73 08:44:47 AM all 18.87 0.00 68.59 1.76 0.00 10.77 08:44:52 AM all 22.58 0.00 69.50 1.40 0.00 6.52 08:44:57 AM all 20.44 0.00 68.39 1.90 0.00 9.27 08:45:02 AM all 22.35 0.00 66.35 1.17 0.00 10.13 08:45:07 AM all 22.06 0.95 62.82 2.05 0.00 12.12 08:45:12 AM all 17.86 0.00 65.69 0.61 0.00 15.84 08:45:17 AM all 20.41 0.00 63.54 0.11 0.00 15.94 08:45:22 AM all 19.41 0.00 58.43 0.02 0.00 22.14 08:45:27 AM all 19.28 0.00 61.03 0.17 0.00 19.52 08:45:32 AM all 17.03 0.00 64.64 0.67 0.00 17.66 08:45:37 AM all 14.26 0.00 64.66 0.10 0.00 20.97 08:45:42 AM all 18.03 0.00 66.56 0.10 0.00 15.31 08:45:47 AM all 18.23 0.00 65.75 0.24 0.00 15.78 08:45:52 AM all 15.08 0.00 66.38 0.04 0.00 18.51 08:45:57 AM all 16.22 0.00 65.87 0.50 0.00 17.41 08:46:02 AM all 20.03 0.00 69.77 0.25 0.00 9.95 08:46:07 AM all 25.62 0.00 68.17 0.25 0.00 5.96 08:46:12 AM all 22.91 0.00 72.83 0.51 0.00 3.74 08:46:17 AM all 23.35 0.00 70.30 1.59 0.00 4.76 08:46:22 AM all 24.97 0.00 67.41 0.80 0.00 6.82 08:46:27 AM all 24.46 0.00 61.03 0.81 0.00 13.70 -- AdminServer start shutdown (User name matches. Shutdown permitted.) 08:46:32 AM all 20.55 0.00 56.72 2.50 0.00 20.22 08:46:37 AM all 22.45 0.00 53.39 1.66 0.00 22.50 08:46:42 AM all 21.82 0.00 52.73 0.95 0.00 24.50 08:46:47 AM all 25.16 0.00 52.20 2.09 0.00 20.56 08:46:52 AM all 21.88 0.00 54.79 1.44 0.00 21.88 08:46:57 AM all 21.77 0.00 57.61 1.69 0.00 18.93 08:47:02 AM all 23.82 0.00 53.88 2.48 0.00 19.82 08:47:07 AM all 19.42 0.00 52.92 3.46 0.00 24.20 08:47:12 AM all 15.27 0.00 52.88 3.47 0.00 28.39 08:47:17 AM all 14.95 0.00 52.15 3.67 0.00 29.22 08:47:22 AM all 18.41 0.00 52.66 0.31 0.00 28.63 08:47:27 AM all 18.54 0.00 52.10 0.98 0.00 28.38 08:47:32 AM all 15.33 0.00 51.86 0.65 0.00 32.17 08:47:37 AM all 14.92 0.00 51.35 0.60 0.00 33.13 08:47:42 AM all 18.06 0.00 53.32 1.43 0.00 27.19 08:47:47 AM all 19.21 0.00 45.29 1.30 0.00 34.20 08:47:52 AM all 15.68 0.00 45.21 1.74 0.00 37.36 08:47:57 AM all 16.00 0.00 47.88 1.55 0.00 34.57 08:48:02 AM all 14.24 0.00 41.73 0.41 0.00 43.63 08:48:07 AM all 16.19 0.00 46.97 0.42 0.00 36.42 -- AdminServer end shutdown (OpenEdge Management cache database closed.) 08:48:12 AM all 11.28 0.00 34.34 4.27 0.00 50.11 08:48:17 AM all 4.85 0.00 8.11 0.84 0.00 86.20 08:48:22 AM all 2.86 0.00 7.02 0.13 0.00 89.99 08:48:27 AM all 4.74 0.00 6.88 1.69 0.00 86.68 08:48:32 AM all 4.04 0.00 6.86 0.84 0.00 88.26 08:48:37 AM all 4.72 0.00 7.21 0.59 0.00 87.48 08:48:42 AM all 7.05 0.00 6.90 2.56 0.00 83.49 08:48:47 AM all 4.22 0.00 6.89 0.61 0.00 88.28 08:48:52 AM all 5.65 0.00 7.07 0.37 0.00 86.91 08:48:57 AM all 4.76 0.00 7.03 0.20 0.00 88.00 08:49:02 AM all 8.77 0.00 6.99 1.06 0.00 83.18 08:49:07 AM all 13.72 0.00 7.99 1.41 0.00 76.88 08:49:12 AM all 10.30 0.00 7.33 0.75 0.00 81.62 08:49:17 AM all 6.85 0.00 2.43 2.60 0.00 88.12
We tried this recommendation (allowDiscovery=false) http://knowledgebase.progress.com/articles/Article/adminserver-using-high-amounts-of-cpu-and-network-resources-at-startup
It did not help.
Are there any ways to find out why AdminServer uses a lot of CPU?
Regards,
Valeriy
Hello,
We solved the problem by updating the OS from SLES 11.1 to SLES 11.3. After that the load on the server has stabilized.
Thank you all for your answers! I really appreciate it.
Regards,
Valeriy
Use $DLC/jdk/bin/jstack <pid> | somefilea few times a few seconds apart and compare the output to see what is running
There is no jstack in my $DLC/jdk/bin. Does it come standard with OE or it need to be installed there? I have OE Advanced Enterprise Edition.
jstack is a java utility that is part of the standard JDK distribution. If you're missing it...
Well, from within OEE you can run /diagnostics/threaddump.jsp which is pretty much the same output.
If diagnostics/threaddump.jsp doesn’t work it will be because the webserver isn’t responding, in which case you need to use jstack instead.
As a matter of interest, were the cachedb properly closed before restarting? Do you see having to run WAL (a bit like bi recovery on OE) and rebuilding indexes on these databases when you restart the adminserver?
In 11.5 Fathom uses more CPU. If you don't really use it. Please bring down fathom.
Hello,
We solved the problem by updating the OS from SLES 11.1 to SLES 11.3. After that the load on the server has stabilized.
Thank you all for your answers! I really appreciate it.
Regards,
Valeriy