Re: proposal: make config parsing non-blocking.

On 4/26/06, Robert Collins <robertc@robertcollins.net> wrote:
> On Wed, 2006-04-26 at 07:23 -0300, Gonzalo Arana wrote:
> > Hi,
>
> perhaps we should discuss this on list ? (You took it offlist, I just
> followed suit).

Absolutely! My mistake, sory (I'm used to have 'Reply-To' headers set
in mails I receive from mailing lists).

> > On 4/25/06, Robert Collins <robertc@robertcollins.net> wrote:
> > > On Tue, 2006-04-25 at 09:40 -0300, Gonzalo Arana wrote:
> > > > I may be wrong, but the only signifficant difference (in wall clock
> > > > time of blocking squid) between checking configuration and applying it
> > > > are the external helpers (external_acl, auth helpers, piped loggers).
> > > >
> > > > Getting configuration data from the network could be a nice thing on
> > > > the other hand, but external_acl provide the means for doing something
> > > > similar.
> > > >
> > > > Rather than providing a 'slow migration' for every configuration
> > > > directive, making 'slow migration' for 'expensive' directives (like
> > > > external_acl, auth helpers, etc.) would have the same result but with
> > > > less work, right?
> > >
> > > Nope.
> > >
> > > Any non blocking task can only be used by other non blocking tasks, all
> > > the way out the main event loop.
> > >
> > > Put another way, if you imagine a call stack:
> > > main()
> > > event_loop()
> > > FUNCTION()
> > >
> > > if FUNCTION is a non blocking task, such as doEvents(), or
> > > comm_select(), then it can call either blocking calls (for a performance
> > > penalty), or other non blocking calls (which is ideal).
> > >
> > > if FUNCTION is a blocking task like squidReconfigure(), then it can only
> > > call other blocking calls.
> > >
> > > So the *first* blocking call means that all other calls down from there
> > > need to be blocking.
> > >
> > > We cannot use any non blocking calls in the reconfiguration process
> > > while the configuration is a blocking event.
> >
> > While I understand your point, I don't understand why does this
> > contradicts what I suggested. My proposal was something like this:
> >
> > 1) reconfigure is requested
> > 2) parse configuration & apply non-slow-migration directives.
> > 3) open new passive socket (if port is changed).
> > 4) assign new callbacks to that port.
> > 5) mark all helpers as 'die & create a new one when idle' (support for
> > this flag has to be added, I think).
> >
> > This way, applying new configuration is a blocking procedure, but
> > should be much faster than waiting for all external helpers to warm
> > up. This is a way to achieve what you suggested, am I right?
>
> Not really. For instance: one common configuration of squid is to have
> the ACL's configured outside of the squid configuration, and reading
> these can take some time.

right.

> Anyhow, what I thought you were saying was to have some slow sections
> within the parsing, which is where my statements did contradict you -
> because the parsing then has to be slow (as opposed to grouping the
> actions to be taken after parsing as fast/slow).

I see, and I agree with you.

It would be just great to have squid reconfiguration without closing
passive sockets, but any deep rework on squid3 will delay it's stable
release. My counterproposal was just to suggest a way of achieving
this with minimum changes to squid3.

Regards,

--
Gonzalo A. Arana