Technical background information on .NET bridge

Have you ever tried just attaching the VS debugger to a process running that .Net code and examine everything (ie. callstacks and memory)?  I would guess that you could learn quite a lot.    And then you could post whatever you wanted because you discovered it all independently ;).  You might also try using the Progress/Telerik JustDecompile to open any interesting .Net assemblies that aren't part of the Framework itself.  I would guess they are not going to be obfuscated, and you would probably be able to make some sense of it .  That depends on what you are looking for.

The integration is probably comparable to how progress integrates with java.  For example, with the progress "JMS adapter" you can send and receive messages to a remote JMS-compliant broker.  I think Progress launches a "symbiont" java process on the local machine and interacts with it via IPC.   In truth they probably don't WANT anyone to have the underlying details about how it works.  It is easier to change the implementation details when nobody else needs to be in the loop.

The integration with the .Net bridge is probably even more efficient, than how they integrate with java.  They can probably launch a .Net appdomain within the same process.  It will be interesting to see how they decide to integrate with .Net core one day.

While I am a .Net developer myself, I haven't ever used it from ABL because our OE installations have always been running on HP-UX.  That will change soon, since HP-UX is end-of-life and OE 12 won't be certified on that platform anymore.

I haven't tried attaching the VS debugger, but without having the sources, will that be of any use?

Decompiling is, afaik, not permitted by the eula, but let's say I heard that the assemblies are obfuscated...

What I did do is examine issues using a .NET memory profiler, which also gives some insight.

Also creating a StackTrace object and showing its contents reveals some things (showing that quite some reflection is at play), but using that to reverse engineer the code is not the easiest thing when you have no further hints...

I'm not expecting the actual implementation of it all, but a decent overview of the components and how they are interacting would be a great start, combined with what's already known.

Thanks Peter.

There's a lot of interesting info in there, though we're missing a presenter to add the details.

I assume the notes are Laura's side notes while presenting?

Some raise more questions (like slide 46: "Only if they ask!" ;-))

I noticed Laura's comment on slide 34: "Internals of this is a whole talk unto itself"

If such a talk/presentation exists, I would be very interested in seeing it!

If you're reading this Laura... (if you're not yet fed up with the support cases we've been throwing your way...)

What I'm currently wondering: for hybrid objects a proxy is generated at runtime, using reflection.

Could it be an performance improvement if that was part of the compilation step?

So that compiling ABL code results in r-code + one or more .NET assemblies that contain the types that now end up being created in a dynamic assembly at run-time?

Or is this step negligible in terms of performance?

Does this generation happen for every use of a specific type, or only once per session?

First, this works nothing like the JMS adapter.  Without getting into the details, the .NET framework is loaded into the process so that we can just make calls into it.  

Yes, for hybrid objects, the proxy is generated at run-time.  Yes, we use reflection for making all calls to the methods/properties that the ABL is trying to reference.

We did think about generating assemblies for the proxy types during compilation but thought that this presented its own problems in terms of file management and deployment.  But we never tried to do it (or to simulate it manually) to compare the performance.  As usual, there were constraints on our time and we could not go down all paths.  I’m sure it would save some time, but what percentage of the time is the question,  We only create the proxy type once per class type.  So once it’s done, it’s done.  

Executing code via reflection also takes more time I believe.  But to avoid that, we’d also have to generate not just the proxy type, but code that calls it and store in the r-code the # of the method to call... or something like that!  

Hi Laura,

Thanks for taking a moment to reply.

I think I have an idea on how things work for hybrids, from a high level at least...

What I'm still wondering is how pure .NET objects are interacted with.

As I understood from a previous support case note (which I suspect came from you), when using a pure .NET object from ABL, there is no ABL counterpart of the object. There is also no proxy. So something must be created on the .NET side to prevent the GC from picking up the object immediatelly. That must be a GCHandle, stored in a collection (_handlemap).

So accessing methods/properties of that .NET object goes directly to the .NET side and back?

How about event subscription? There must be some "EventSubscriber" object in the bridge to call the ABL side handler?

Please correct me if I'm wrong.

Regards,

Lieven

By using a memory profiler I think I can see what happens when subscribing to .NET objects events.

A new object "<FullyQualifiedTypeNameWithoutDots><EventName>Receiver" is created (e.g. "SystemWindowsFormsFormDisposedReceiver") to delegate the event handling to the ABL side. Correct?

You're doing a very good job of figuring it all out!

Correct, for a pure .NET object there is no ABL counterpart of the object. There is also no proxy.

Yes, we create a GCHandle, which is stored in a collection (_handlemap) to make sure the object stays around. We store it in the map so that we never create more than one GC handle for any specific .NET object.  When the object is no longer being used from the ABL, we free it.

Yes, accessing methods/properties of that .NET object goes "directly" to the .NET side.  Directly is in quotes, because there are a couple of layers it goes through. And of course we often need to do data conversion when going there and back.

And again correct, we emit a an event receiver that .NET can call when the event is fired.  That in turn calls back to the ABL to run the handler.

You're doing a very good job of figuring it all out!

Correct, for a pure .NET object there is no ABL counterpart of the object. There is also no proxy.

Yes, we create a GCHandle, which is stored in a collection (_handlemap) to make sure the object stays around. We store it in the map so that we never create more than one GC handle for any specific .NET object.  When the object is no longer being used from the ABL, we free it.

Yes, accessing methods/properties of that .NET object goes "directly" to the .NET side.  Directly is in quotes, because there are a couple of layers it goes through. And of course we often need to do data conversion when going there and back.

And again correct, we emit a an event receiver that .NET can call when the event is fired.  That in turn calls back to the ABL to run the handler.

Thanks, it's great to get confirmation from the expert ;-)

In a technical explanation on a support case you (I guess it was you) once told that when there are no more references from the ABL to a hybrid, you can't immediatelly GC it because the .NET side could still call back. To make it go away, you emit a finalizer in the proxy that can call back to the AVM and do the job.

You made a remark that there is a slightly different mechanism for Forms.

Could you elaborate on the difference?

The Dispose pattern causes  GC.SuppressFinalize to be called.  Therefore for any object implementing IDisposable (not just forms), the finalizer will never run.  See this article:

stackoverflow.com/.../why-does-the-traditional-dispose-pattern-suppress-finalize

Therefore we needed a different way.  That way is that once Dispose is called, if the ABL no longer has a reference to the object, we will delete it.  We've fixed a couple of bugs recently with the order in which things happen on Dispose.  There were cases where we deleted the object prematurely - before Dispose was called on all sub-components, for example.  But hopefully, that should all be good now (as of 11.7.3.x, 11.7.4)

Then that must be one of the reasons why overriding Dispose() in a hybrid is not allowed?

Does the proxy get an override for Dispose(), first calling base:Dispose(disposing) and as last step calling back to the ABL to have things cleaned there?

If it was overridable, then calling SUPER:Dispose(disposing) would have to be the final statement in that method, or else you would mess things up pretty badly (no GC of the object would happen).

If Dispose() is not called during the lifetime of the object, then the finalizer will ensure it gets called eventually*, and that could happen on another thread, so that's probably another reason not to allow overriding it (knowledgebase.progress.com/.../Why-is-the-Dispose-method-FINAL-in-ABL)

See also docs.microsoft.com/.../dispose-pattern

We've had some Dispose related issues fixed, but I'm afraid we've recently logged a new one (00463658), which might end up on your plate sooner or later...

* At least for any object inheriting Component, which is all Controls and Forms... https://referencesource.microsoft.com/#System/compmod/system/componentmodel/Component.cs,053ea5fbb05453a0

[quote user="Laura Stern"]

Therefore we needed a different way.  That way is that once Dispose is called, if the ABL no longer has a reference to the object, we will delete it.  We've fixed a couple of bugs recently with the order in which things happen on Dispose.  There were cases where we deleted the object prematurely - before Dispose was called on all sub-components, for example.  But hopefully, that should all be good now (as of 11.7.3.x, 11.7.4)

[/quote]

I believe those were the doomed 15580 errors when closing screens (at runtime or in the Visual Designer)?

Re: Then that must be one of the reasons why overriding Dispose() in a hybrid is not allowed?

No, I don't think that is related to why it's FINAL.  I believe that is only because it might be called on another thread, as you later referenced.  

And no, the proxy does not call back to the ABL to do Dispose().  That would have the same threading issue. If you want to do something on Dispose in the ABL you should subscribe to the Disposed event.  We and others had a similar conversation some time ago on the forum :-)

community.progress.com/.../38990

And yes, Mike, it is possible that the 15580 error was due to the bug we just fixed.

Hmmm, but the proxy is a derived class of the .NET class the ABL hybrid wants to inherit, right?

I'm not seeing it in my memory observations using a .NET memory profiler, so I can only guess...

Correct - i.e., if the ABL class inherits from System.Windows.Forms.Button, the proxy also inherits from that.

You should be seeing proxy objects in the .NET profiler.  

I see it now, it was hidden in plain sight with the same name as the ABL class...

:-)