Re: benchmarking squid on solaris/x86

On 20 Mar 2002, at 2:34, Adrian Chadd <adrian@squid-cache.org> wrote:

> On Tue, Mar 19, 2002, Duane Wessels wrote:
>
> > I've tried the obvious things: file descriptor limits, ephemeral
> > port range, ...
> >
> > Does anyone have experience with solaris on x86? I'm trying to
> > decide if its an inherent limitation, or should solaris be able to
> > keep up with linux on the same hardware?
>
> Uhm, is it swapping lots? I don't know about the solaris 8 pager,
> but solaris 7's acted like FreeBSDs - swap out chunks of processes
> in case RAM is required in a hurry.

 For Sol7, you need kernel patch 105182-9 or better, and then set
 in /etc/system:
set lotsfree = 2048
set priority_paging=1

 by default, Solaris is keeping abit too little ram free, and spikes
 or ram allocation cannot be satisfied from freelist. lotsfree ups
 this to 2K pages. priority_paging turns on new allocation scheme
 where OS distinguishes between cache pages and proc pages. It
 uses 2x lotsfree pages in "almost free" state and grabs those from
 FS cache. Those pages are actually used as cache, but ready to be
 stealed when ram allocation is needed.

 Basically, these two stops Solaris from pageing out parts of
 processes when there is temp shortage of ram.
 NB! on Solaris8 "priority_paging" should NOT be used. Sol8 has
 changed vm handling again and they claim this is a nonissue now.

set tcp:tcp_conn_hash_size = 8192 # your max openfiles
 default is 256. I suppose this helps when there are more than
 256 open tcp sessions.

set tune_t_fsflushr = 2
 set OS to check more frequently for dirty pages not flushed.
 spreads pageout load in time abit. less dirty pages are flushed
 in one go - no spikes of disk overload.

 changes in /etc/system require a reboot. make sure you have a
 backup copy of /etc/system! in case you screw it you won't be
 able to boot up and would need to specify alternate file at
 boot prompt.

 TCP stack.
ndd -set /dev/tcp tcp_mss_def 1460
ndd -set /dev/tcp tcp_xmit_hiwat 65536
ndd -set /dev/tcp tcp_recv_hiwat 65536
 upp tcp buffers (recompile squid), and prefer large segments.

ndd -set /dev/tcp tcp_rexmit_interval_min 1000
ndd -set /dev/tcp tcp_rexmit_interval_initial 2000
ndd -set /dev/tcp tcp_rexmit_interval_max 20000
 Avoid excessive retransmit overhead. Preference for WAN.
 LAN performance will be slightly lower in case of drops.

ndd -set /dev/tcp tcp_time_wait_interval 10000
ndd -set /dev/tcp tcp_fin_wait_2_flush_interval 47500
 these two are consuming kernel resources and needs constant
 checks thus slowing down cpu. defaults are way too high for
 really high-performance cache.

ndd -set /dev/tcp tcp_smallest_anon_port 8192
 upp ephemeral port range. Default together with time_wait
 timings and high tps rate can create shortage of these ports.

 those changes activates immediately, but you need to restart
 apps.

 Oh, make damn sure you use dlmalloc on solaris. Solaris' own
 malloc is complete crap.
 You might also get better performance by using gnuregex instead
 of native libs. Generally, Solaris' support libs are unoptimised
 for latest Pentiums. memory/string copy optimised versions can
 make a difference.
 Make sure you compile with -O2 (squid's configure turns that off
 for solaris, dunno why) and -march=pentiumpro

> Also, remmember that Solaris _is_ compiled for SMP performance
> out of the box, so things will be slower on a single CPU.

 True, but performance difference noticed is abit too sharp.
 Under high stress like >2000 concurrent sessions I'd expect
 things to be other way round...

------------------------------------
 Andres Kroonmaa <andre@online.ee>
 CTO, Microlink Online
 Tel: 6501 731, Fax: 6501 725
 Pärnu mnt. 158, Tallinn,
 11317 Estonia
Received on Wed Mar 20 2002 - 03:37:44 MST