Reference Counting Question

[Frederick J. Harris]

I have a question about reference counting as related to passing interfaces by value through function parameters.  And my question relates to the problem Steve Pardoe was/is having with the massive and complicated CQG Library in CQGCEL-4_0.dll.  That library has an outgoing interface in IDL notation as follows ...

dispinterface _ICQGCELEvents

One of the event procedures of that interface is InstrumentDOMChanged....

Code Select Expand


[id(0x00000006), helpstring("Fired whenever instrument DOM is updated")] HRESULT InstrumentDOMChanged
(
[in] ICQGInstrument* cqg_instrument,
[in] ICQGDOMQuotes* prev_asks,
[in] ICQGDOMQuotes* prev_bids
);


Note that all three of the parameters are interfaces passed by value, which in C or C++ speak resolves to V Table pointers to the various interfaces.  From what I gather, Steve's first use of this event procedure in PowerBASIC looked something like this ...

Code Select Expand


METHOD InstrumentDOMChanged <6> (BYVAL cqg_instrument AS ICQGInstrument, BYVAL prev_asks AS ICQGDOMQuotes, BYVAL prev_bids AS ICQGDOMQuotes)
  CONTROL SET TEXT hDialog, 1006, STR$(cqg_instrument.DOMBids.Item(0).Price)
  ' Possibly do other stuff with interfaces
END METHOD


Steve noted that his program was accumulating a tremendous memory overhead the longer it ran, and this only seemed to be occurring with PowerBASIC clients.  The memory drain seemed to diminish tremendously if Steve put object.Release() calls within the procedure for the three interfaces like so ...

Code Select Expand


METHOD InstrumentDOMChanged <6> (BYVAL cqg_instrument AS ICQGInstrument, BYVAL prev_asks AS ICQGDOMQuotes, BYVAL prev_bids AS ICQGDOMQuotes)
  CONTROL SET TEXT hDialog, 1006, STR$(cqg_instrument.DOMBids.Item(0).Price)
  IF ISTRUE gUseRelease THEN
     cqg_instrument.Release()
     prev_asks.release()
     prev_bids.release()
     LOCAL RC AS DWORD
     CONTROL SET TEXT hDialog, 1005, STR$(RC)
  END IF
END METHOD


Now, it would be my understanding in a 'garbage collected' language like PowerBASIC (and any version of BASIC, for that matter), these release calls shouldn't have to be made, because the compiler should auto-generate clean up code to release the three interfaces passed by value.  When Steve informed me how this was working out I immediately suspected that PowerBASIC might be failing to do the release calls in this unusual circumstance of event procedures receiving interface pointers by value.  I knew for absolute certain that PowerBASIC faithfully does Release() calls on non-event procedure functions where interface parameters were involved, because I had tracked these calls countless times in my debugging code where I had created COM servers with low level C or PowerBASIC code.

So I set about creating a low level simplified mock up of this type of app where several interfaces would be passed by value (V Table pointers) back to the client through an outgoing interface (Event Procedures).  So here is my question, which is kind of a statement too, because I believe I know the answer.  But I want to be absolutely clear about it. 

When a server passes an interface pointer back to a client through an event procedure parameter, or any way at all for that matter, that operation constitutes a 'copy' of the interface pointer, and therefore requires an AddRef() be called on that pointer?  Now, I personally believe that statement to be true.  Consider though that in Steve's case, the server is a C++ object where automatic reference counting doesen't take place - one does AddRef()'s and Release()'s manually.  So, right in the C++ source code there would have been an explicit AddRef() call on each of those three interfaces.  Further realize that in whatever procedure in the C++ server those AddRef()'s took place, there would not have been Release() calls on those three objects, because it would be the client's responsibility to Release() those objects created for it.

Now, the plot thickens somewhat when we turn to the PowerBASIC client receiving those three AddRef()'ed interface pointers.  My understanding of what should happen there is that the PowerBASIC client (or a client written in any language, for that matter) only needs to do Release()'s on those pointers.  They have already been AddRef()'ed by the server in preparation for their consumption by the client.  The client Release()'s when finished?

Now, is there any part of the above which I have wrong?  To make myself absolutely clear, let's use real numbers.  Let's say the server's state at the beginning of whatever procedure it is which triggers the event is at a reference count of 3.  It calls QueryInterface() internally three times to generate the three interface pointers it is going to send to the client through its outgoing interface.  This will bump the reference count up to six (QueryInterface(), remember, AddRef()'s pointers returned by it).  The client receives the three AddRef()'ed interface pointers, and when finished with them calls three Release()'s which brings the reference count back down to 3, which is where it started.  In the case of PowerBASIC, which does automatic reference counting, the Release() calls should have been generated by the compiler, and they should execute after the event procedure terminates.

Again, is there any part of the above which I have wrong?  The reason I'm stressing this is that the above behavior isn't what I'm seeing in my mock up code.  What I'm seeing is that the PowerBASIC client is indeed doing the Release()s on the interface pointers passed to its event procedure, but it is first AddRef()'ing them!  The net result of this is that there is no change to the reference count after the PowerBASIC event procedure exits, and it shouldn't be the C++ server's responsibility to Release() those pointers which it passed out.  If the server would have QueryInterface()'ed for those pointers and used them only internally and not passed them out, then it would have been its responsibility to Release() those pointers when finished.  But in passing them out to a client it would be absolved of further responsibility for those outstanding references.

I would be interested in anyone's 'take' on this.  I do have both server and client code I could post on this, but it isn't ready yet.  I'm really just trying to sort it out myself at this point.       

[Frederick J. Harris]

So I guess what it really amounts to is the question of whether the server should call Releases on the three pointers it AddRef()'ed.  In terms of an arguement in favor of that, one could note that it still has use of the pointers within the procedure it QueryInterfaced for them. 

[Frederick J. Harris]

So, using my above example, the reference count starts at 3.  The server generates another 3 by creating the soon to be parameters.  Now we're up to 6.  When the parameters are received in the client it AddRefs them, which brings us to 9.  After the event proc in the client terminates, three releases bring us to 6.  Then, back in the server the procedure terminates which triggered the event, and three more Releases are called which brings the Rewf Count back to the original three.  I suppose that might be how it should go.  But if so, how come those extra Releases in Steve's app seem to solve the problem???

[José Roca]

An event method is just a function and, as any PB function, when it receives an interface reference by value it AddRefs it to make sure that the pointer is valid during the life of the method/funtion and not invalidated by being the interface released elsewhere, and Released when you set it to NOTHING or goes out of scope. If the component that passes the interface pointer by value delegates to you the responsability to release it, it must told you this in the documentation.

There are also plenty of API functions or COM methods that pass pointers to strings or arrays of strings and the documentation must tell you if they are static and you don't have to release them or have been allocated with CoTaskMemAlloc and you must free them with CoTaskMemFree.

There would be a bug in PB if a method AddRefed the interface pointer and later did not relased it, or if it created a temporary string and later did not free it. Otherwise, no.

[Frederick J. Harris]

Yes, that's the conclusion I came to also Jose.  It was sort of forced on me against my will, so to speak.  I was quite convinced my erroneous conception was right, even in the face of some evidence against it.  Oftentimes I have found that simply typing a proposition or argument down helps me think it through. 

Thanks for your thoughts on this.

Fred