Our proxygen operation has gotten extremely slow. Are there any plans to improve it? If nothing else, can't Progress design some of their tooling to allow better use of the CPU cores? It seems to me that they often forget that a modern CPU is not limited to working on one application thread at a time.
I ran "process monitor" and "process explorer" to investigate what bproxygen.bat is actually doing, and I was somewhat shocked at the contorted way in which it generates a proxy dll for appserver.
It starts with a basic java application (com.progress.open4gl.proxygen.Batch). But soon afterwards you see that there is an astonishing number of other processes that are started and stopped for the duration of the operation, including but not limited to :
One of the final operations is to compile all the intermediate results into a appobject assembly. This builds out a massive command line string and sends it to the compiler (csc.exe). We've found that this command line string has a limit (I think it is 32 or 64 KB of characters or something along those lines).
Because of the limit on the command string, the compilation operation will fail depending on the current directory location on disk in which bproxygen.bat is doing its work. For example, if you are working in the folder C:\Data\Workspace\Progress\OpenEdge\MyVeryLongProjectName\AppServer\Proxy\Oemodule\ThisProxy_001.r then the fully qualified directory name will be multiplied endless times and sent to csc.exe in a super-long command string. That causes failures. But if you move ThisProxy_001.r back to a shorter path (c:\Temp\Short\ThisProxy_001.r) then it allows more stuff to be included in the proxy dll.
Is anyone aware of plans to improve proxygen? I am happy that Progress is investing in PASOE (and in the proprietary APSV transport), but the client-side tooling is long overdue for improvement. I'm not sure if we've seen any substantial improvement in Proxygen for many years (maybe ten or more). I'd agree that it is hardly a glamorous thing for Progress to work on, but as a developer I run this numerous times a day and it is a very tedious and time-consuming part of my day. If Progress gave this even a little bit of attention, it would be very much appreciated.
Thanks Brian,
Yes, eventually our proxies get large. A couple of them take over ten minutes to proxygen. Obviously that got way out of hand especially given that, when one of them is being generated, only one CPU core is doing work and the rest of the machine is idle. At least when we run our CI builds, the two proxies can be built from two independent OS processes and use separate cores. But there is still some significant CPU capacity on our build server that is not being utilized for the CPU-bound proxygen operation.
As you indicated, splitting things into smaller proxies is a good strategy but it only goes so far. We already begin to see performance pains after a xpxg grows to include about two or three dozen proc objects (with related internal procedures and a variety of DS/TT parameters). This pain can involve waiting 30-60 seconds for a new proxy assembly.
That may not sound like a lot of time but if you have to repeat that all day long it gets quite tedious and can become one of our productivity bottlenecks. Remember that proxygen needs to be run whenever the DS/TT data structure is changed or any time a method signature is changed (ie. parameters are added or removed). And I would add that in many cases a proxy xpxg is forced to include *all* the proc objects which reference a shared dataset definition - this places a limit on our ability to split things up.
Thinking out loud for a minute, it doesn't seem too difficult to imagine a proxygen that would do all the necessary DS/TT work ahead of time, in parallel, and then proceed to do all the proc object work in parallel too. As things work today, much of my machine's CPU is idle while we're waiting for these proxy assemblies to be created. There is a huge opportunity for improvement. I hope I'm not the only one complaining. (Now that PASOE offers other options like REST, it probably lowers the prioritization of proxygen improvements, and it will also shrink the community of developers who are trying to use proxygen. )
Another thought... if Progress doesn't have the necessary developer resources to work on these types of things then perhaps they could open source some of their developer tools such as this old proxygen utility. I'm sure that there are .Net or Java programmers who would be happy to rearrange some of this code and send it to multiple concurrent threads. Nowadays that is a pretty standard thing to do (at least outside of the world of ABL).