Re: aio_read implementaion in Squid

On Wed, Sep 12, 2001, Venkatesh wrote:
> Hello joe,
>
> I was asking this for Net I/O. I had a cursery look on eventio and signal
> driven io branches. eventio branch also was done by Henrik and adrian (If I
> am right). I already asked adrian about eventio, but i did not get any
> reply. It seems he is busy with his work now. I got it what you are telling.
> But i want to know if we go to aio for the Net I/O, what are the issues we
> have to face?

I'm busy, and I'm still trying to get my net.presence back online.
I'm replacing my creative.net.au box tomorrow, which should make everything
much nicer for me (and accessible and resolvable!)

> I do not know you are remembering or not. I had asked you many Qs about CPU
> Usage. Now my objective is change the poll in Squid. I am looking for very
> high performance Squid. I want to get feedbacks from you all before i
> proceed. And I am very curious to know about the OS compatibilities problem,
> If we go for aio.

Right. Don't use aio_read/aio_write for network IO. I did this a year
or so ago for Solaris/FreeBSD, and the results aren't as good as you'd
think they would be. Reason: the posix AIO routines are designed for disk
IO, not socket IO. They work for socket IO, but you'll get a lot of subthreads
handling your network IO. Its very inefficient.

The current idea on the table is a reimplementation of the comm code
and its users to use a comm_read() and comm_write() style callback
interface - comm_write() exists, but comm_read() doesn't. Once we know
when data is expected (rather than just blindly registering for read
IO completion and throwing them away via deferred reads) a much,
much more efficient poll/kqueue/etc-based network IO model can be
used.

The commloops/eventio branches are kind of the right place to start, but
I suggest looking at comm_write(), implementing a similar comm_read()
scheme (take a buffer, a fd, a size, a callback and a callback data
pointer, and call a read completion callback when the read completes
or errors, just like comm_write(), but wihtout the buffer free() ;-)
and then looking at the http server and client code (http.c and client_side.c)
where read()/FD_READ_METHOD() is called. Replace these calls with calls
to comm_read(), throw away the deferred read code, and then come back with
your code. :-)

I'm still out of the loop with coding. Sorry. :(

HTH,

Adrian
Received on Wed Sep 12 2001 - 04:01:57 MDT