Re: Architecture Overview

On Tue, 2008-08-26 at 02:23 +1200, Amos Jeffries wrote:
> Okay, got the pretty-picture drawn up.
>
> NP: this is drawn up as a high-level flow from my accumulated view of
> all our work to date and where its heading. That includes Adrian's
> Squid-2 work and where I see it most efficiently mapping into Squid-3.
>
> It should be very similar to how squid currently works. With a few major
> differences that we have all spoken and planned things around already.

Thank you for working on this picture. I am not quite sure I interpret
it correctly, but I do see a few distinct objects there: Data Pump, HTTP
Parser, and Store. This is more or less clear, at least at this high
level.

It is not clear to me whether the other blobs such as Protocol
Processing and Protocol Handling are flows, objects, or something else.
I am also not sure whether the arrows represent passing message data,
passing processing responsibility, or something else. Do different
colors and blob shapes mean something?

If we want an architecture picture, I think it would be great if we can
formulate it in terms of objects and flows among them. This should make
roles and boundaries much more clear.

> 1) Data Pump side-band. This is a processing-free 'pump' for any
> requests which do not actually need to go through Squid's twisted
> logics. With all logics compiled out it should be equivalent to a NOP.
> But can be used by things such as adaptation components as an IO
> abstraction.

What data does the Data Pump pumps? Message bodies? What are the valid
ends of a pump? Can there be many Pumps per HTTP transaction? Does the
Pump communicate any metadata to the other side?

If adaptation components can use Data Pump as an I/O abstraction, should
not all other high-level components processing the transaction do the
same so that high-level I/O code could be reused among all the
components?

The NOP equivalence mentioned above confuses me. Do you mean that the
pump does not copy data if it does not have to?

> 2) Client Facing IO is unwound from all processing logics. It's simply a
> raw input layer to accept connections and interface to the clients.

The "external" side of the Client Facing IO blob is socket API and such,
right? What is the Squid-side interface of the Client Facing IO blob? A
collection of portable socket-level routines? Some kind of a Transaction
object?

Is limiting the number of accepted connections a "processing logic" or
"Client Facing IO" logic?

> 3) Server Facing IO is likewise unwound from processing logics AND from
> client IO logics. Though in reality it may share lowest level socket code.
>
> 4) Processing components are distinct. Each is fully optional and causes
> no run-time delay if not enabled.

What decides which processing components are enabled for a given
transaction? Do processing components interact with each other or a
central "authority"? What is their input and output? Can you give a few
examples of components?

The text on the picture seems to imply that there can be only one
Processing Component active for a given transaction, which worries me,
but perhaps I just do not understand what kind of Components you are
describing here.

> 5) Stores are an optional extra, if the configuration calls for caching.
> But not needed for basic operations.

Is there a single global index of stored responses? If yes, is it
enabled only when caching is enabled?

Do you consider request merging a form of caching?

You have not mentioned the Control Logic blob. Is it pretty much the
same as the "Processing Components" blob? What does it control?

> If we all agree and work towards this type of model and things are kept
> modular isolated to the highest levels. I don't see the future
> integration of either squid branch or CacheBoy as being a big task.

I think we would need a more detailed or precise architecture
description to be able to "work towards it" or, more precisely, to
identify code that does not satisfy the architectural constraints.
Otherwise, everybody will be claiming to conform to the Architecture
principles but there will be no improvement as far as merging Squid2 or
external code into Squid3.

BTW, the text descriptions you gave above appear much more useful than
the picture itself. Perhaps we can define the main objects and flows
better and then redraw the picture to match the descriptions? Should
this go into a wiki?

Thank you,

Alex.
Received on Mon Aug 25 2008 - 20:03:41 MDT