refcounting models - an update

I've written some thoughts on refcounting + cbdata and interoperation,
both here and in the dev guide.

I've finally had time to sit down and check for corner cases - so here
is a thumbnail sketch of my guidelines. I'll update the developers guide
shortly... if no-one poses a case I hadn't considered.

As a refresher, in 3.0 we have 3 ways memory can be allocated:
* via xfree and xmalloc - ::operator new and ::operator delete use
this.
* via MemPools
* via CBDATA (which uses MemPools).

In terms of remote storage - objects with pointers to other objects, we
have 4 models:
* plain pointers
* cbdata pointers
* RefCount<> template smart pointers. (new in 3.0)
* Manually refcounted pointers. (i.e. StoreEntry).

What I'm here to address is guidelines for choosing which allocation
strategy, and which remote storage strategy to use when extending (or
cleaning up) squid.

There are three common usage patterns of remote storage that we see in
squid:
* Complete ownership
* shared ownership
* callback ownership

Complete ownership:
When object A owns an instance of class B, and is the only object that
ever keeps that reference beyond synchronous calls, A is completely
responsible for the management of B - when it is allocated, and when it
is freed. In this case, either the default operator ::new and ::delete,
or a MemPool for class B established for memory allocation. A plain
pointer is most appropriate for the remote ownership. Note that this is
-not- compatible if the instance of class B is ever passed (or passes
itself) to a callback function, where cbdata should be used - see below.

Shared ownership:
When multiple objects hold references to a single instance, then
RefCounting (manual or automatic) is appropriate. This allows the
original creator to be disposed of and still have the instance intact
and available for the other objects that hold references. Note that this
is -compatible- with Callback ownership. Additionally, callback
ownership is -not- appropriate to use if one wants to call methods on
the object. That requires reference counting, which is -not- what cbdata
provides.

Callback ownership:
Squid's callback logic works thusly:
When callback is ready to occur, the callback is only called IF the
object to be called back is still valid - it hasn't been destroyed. The
CBDATA functions such as cbdataReferenceValid are used to check this. An
object that holds function,data pairs has callback ownership over the
data - that is:
* It doesn't prevent the data object being freed.
* It doesn't hold a reference to the object.
* It is able to test for the objects validity.
Callback ownership requires the objects memory to be managed by the
cbdataAlloc and cbdataFree routines. This is compatible with Shared
ownership.

Ok, now the guidelines:
* If an object calls any callback routines, it must be CBDATA, and have
it's new and delete operators overloaded appropriately. Additionality it
must have a deleteSelf() method.
* If an object has more than one owner (that is, an object that either
calls into it, or that passes it along to another object), then it
should be automatically reference counted. Manual reference counting is
deprecated.
* If an object has only one owner, then it doesn't need to be reference
counted, but may still need to be CBDATA - see two points up.

Now, in terms of corner cases, Henrik expressed some concern about
cbdata and reference counting interactions:
It works our very simply:
* Anything that needs to call on an object, or use it for anything other
than an argument when doing a callback, holds a RefCount to it. That
guarantees the object is not deleted until the RefCount is set to NULL.
* Callback routines call cbdataReference and cbdataReferenceDone as
usual.
* If the RefCount hits 0 while cbdataReferences are outstanding, then:
  * the destructor runs, freeing the objects resources.
  * cbdataFree is called, marking the object as invalid
  * routines waiting to callback will not do so - as it's invalid.
  * the memory will be held until all cbdataReferences are released.
* If the cbdata references is 0 when the RefCount count hits 0, then the
above happens, but the memory is immediately released.

The only catch is there is likely areas in squid abusing
cbdataReferences as a substitute for refcounting - and they should be
fixed when adding the RefCount machinery to a given class .. like I did
with ConnStateData today. No harm will come to such existing areas, as
the behaviour doesn't change until one adds the RefCount machinery.

Rob

-- 
GPG key available at: <http://members.aardvark.net.au/lifeless/keys.txt>.