Re: [RFC] SMP dependency hell

On 04/02/11 22:31, Tsantilas Christos wrote:
> On 02/04/2011 08:38 AM, Amos Jeffries wrote:
>> Christos,
>>
>> I think the moving the Coordinator to a registry subscription pattern is
>> what we need to do to fix the depdendency hell from SNMP support changes
>> yesterday.
>>
>> The problem stems around src/ipc/Coordinator.* containing
>> handle*Request() handle*Response() for particular components which are
>> it's users. The obvious ones are Snmp req/resp but the cachemgr ones
>> also hit from the inverse side.
>>
>> In the end the SNMP and MGR components should be higher level than IPC
>> and register a Subscription handler/call with it for runtime selection
>> instead of being hard coded at build time.
>>
>> Amos
>
> Is is not so easy. This is only a very small part of the problem. The
> new code uses various external objects and functions, which are located
> to other libraries or objects.
> The Coordinator does not use only SNMP and MGR components but also uses
> the comm_* functions and objects located in comm.cc and Comm* files and
> many other functions and objects.
>
> I was able to bypass testACLMaxUserIP linking problems adding the
> following files:
> BodyPipe.cc DnsLookupDetails.cc tests/stub_fd.cc pconn.cc
> tests/stub_ipcache.cc tests/stub_ssl.cc tests/stub_client_db.cc
> tests/stub_net_db.cc
> and removing the stub_comm.cc

Strange that these would all be needed to add. I am able to build fine
with the existing .cc in trunk and removing several of the so-called
COMMON_LIBS:

@@ -1101,7 +1098,17 @@
  nodist_tests_testACLMaxUserIP_SOURCES= \
          $(TESTSOURCES)
  tests_testACLMaxUserIP_LDADD= \
- $(COMMON_LIBS) \
+ auth/libacls.la \
+ ident/libident.la \
+ acl/libacls.la \
+ eui/libeui.la \
+ acl/libstate.la \
+ auth/libauth.la \
+ acl/libapi.la \
+ base/libbase.la \
+ libsquid.la \
+ ip/libip.la \
+ fs/libfs.la \
          $(top_builddir)/lib/libmisccontainers.la \
          $(top_builddir)/lib/libmiscencoding.la \
          $(top_builddir)/lib/libmiscutil.la \

>
> But now I have worst problems with the testDiskIO (the last test does
> not build).

Like the ACL test above DiskIO does not require any IPC or SNMP actions.
This makes it build clean for me:

@@ -1311,11 +1328,21 @@
          SquidConfig.o \
          CommCalls.o \
          DnsLookupDetails.o \
- $(COMMON_LIBS) \
+ auth/libacls.la \
+ ident/libident.la \
+ acl/libacls.la \
+ eui/libeui.la \
+ acl/libstate.la \
+ auth/libauth.la \
+ base/libbase.la \
+ libsquid.la \
+ ip/libip.la \
+ fs/libfs.la \
          $(REPL_OBJS) \
          $(DISK_LIBS) \
          $(DISK_OS_LIBS) \
          acl/libapi.la \
+ mgr/libmgr.la \
          $(top_builddir)/lib/libmisccontainers.la \
          $(top_builddir)/lib/libmiscencoding.la \
          $(top_builddir)/lib/libmiscutil.la \

>
> The main problem is that all libraries under src/ are not real libraries
> but just objects collections. It does not make sense to try use them as
> libraries.
> Real libraries should have a hierarchical dependency (I am not sure if I
> am using the right words here.)

I understand what you are trying to say, but it is false. These
libraries are as real as any. The difference is a single compiler switch
and a few bytes in the start of the binary.

These libraries should have the same task+information groupings as any
'real' library. With a task each performs layer on top of other tasks
which may be existing in themselves or in another library elsewhere.

The 'glue' code that links two libraries strongly together must go in
the "upper" one. So that the upper layer 'knows' about the lower layer
but the lower only knows itself.

In this case IMO the SNMP and MGR are high layers the IPC is lower yes?

The other objects you mention are separate problems, which may or may
not need to be solved. ie Store pulls in many things but itself is
needed only when MGR is.

>
> In general, I do not know if it is worth to spend all of this time
> trying to make this tests to work :-(

It's worth a little but not too much. Where a small change can fix it do
so. Otherwise stub_*.cc it until a larger refactor can be done.

In the background I'm playing with graphing tools today to see if we can
scan the code and produce a image of the dependencies. To plan a better
long-term fix.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.10
   Beta testers wanted for 3.2.0.4