RE: [Devel] Re: [squid-users] TPROXY on squid-2.6S1

I'm using Debian 3.1 (sarge) with a 2.6.15.6 + cttproxy patch.

I've attached a patch that fixes the 1024 fd bug, an NTLM auth bug, and
allows NTLM auth to work with pipeline prefetching on. These problems
should be fixed in the next squid release.

I would like to add the following to my previous list of requirements for
tproxy + wccpv2:
- You must make sure rp_filter is disabled in the kernel
- You must make sure ip_forwarding is enabled in the kernel

Can you please check that you've enabled ip_forwarding in your kernel. If
that doesn't work, I don't know if the "vhost vport=80" is required in the
http_port line in the squid config (we don't have these options enabled on
our proxies).

I use the ip_wccp module to make the kernel handle the GRE packets correctly
(which works slightly differently from the ip_gre module). Do you have a
GRE tunnel set up in linux? If so, what command are you running to set it
up? I don't have an example to give you here, but I'm sure other people are
using the ip_gre module with wccp to handle the GRE packets, and should be
able to help.

Regards
Steven

> -----Original Message-----
> From: tino [mailto:tino.kriswanto@gmail.com]
> Sent: Wednesday, 12 July 2006 12:53 PM
> To: Steven Wilton; 'Adrian Chadd'
> Cc: 'Kashif Ali Bukhari'; devel@cacheboy.net; 'chima s'
> Subject: Re: [Devel] Re: [squid-users] TPROXY on squid-2.6S1
>
> Hi, Steven,
> Many2 thank for your config & I will immediate hands-on my squid box
>
> May I know your distro & kernel version ? (for shortcut,
> incase, I am using
> fedora4 upgraded to kernel-2.6.15.7 with
> cttproxy-2.6.15-2.0.4 patch from
> balabit )
>
> Based-on cachemgr, at least we need 2000-3000 filedescriptor
>
>
> this is my last config which not work :
>
> I saw wccp hit increments at router, by redirect packet to squid-box .
> Service Identifier: 80
> Number of Cache Engines: 1
> Number of routers: 1
> Total Packets Redirected: 1123
> Redirect access-list: 155
> Total Packets Denied Redirect: 650922
> Total Packets Unassigned: 25043
> Group access-list: -none-
> Total Messages Denied to Group: 0
> Total Authentication failures: 0
>
> Service Identifier: 90
> Number of Cache Engines: 1
> Number of routers: 1
> Total Packets Redirected: 224
> Redirect access-list: 156
> Total Packets Denied Redirect: 206844
> Total Packets Unassigned: 17095
> Group access-list: -none-
> Total Messages Denied to Group: 0
> Total Authentication failures: 0
>
> I saw hit increments in iptables :
> Chain PREROUTING (policy ACCEPT 11517 packets, 2009K bytes)
> pkts bytes target prot opt in out source destination
> 76 24942 TPROXY all -- any any anywhere
> anywhere TPROXY
> redirect 0.0.0.0:3128
>
> But still no hit at access.log, and my host still can't open the web
>
> My last squid-box config :
>
> #iptables :
> iptables -t tproxy -A PREROUTING -j TPROXY --on-port 3128
>
> #part squid.conf :
> http_port 3128 transparent tproxy vhost vport=80
> always_direct allow all
> wccp2_router y.y.y.y
> wccp2_forwarding_method 1
> wccp2_return_method 1
> wccp2_service dynamic 80
> wccp2_service dynamic 90
> wccp2_service_info 80 protocol=tcp flags=dst_ip_hash
> priority=240 ports=80
> wccp2_service_info 90 protocol=tcp flags=src_ip_hash,ports_source
> priority=240 ports=80
>
> #part of my cisco config:
> ip wccp 80 redirect-list 155
> ip wccp 90 redirect-list 156
> int fasteth0 >ip wccp 80 redirect out (gateway to internet)
> int fasteth1 >ip wccp 90 redirect out (my client gateway)
> int fasteth3 >ip wccp redirect exclude in (squid-box attached here)
> access-list 155 permit ip host x.x.x.x any
> access-list 156 permit ip any host x.x.x.x
>
> #modules:
> [root@tproxy sbin]# lsmod
> Module Size Used by
> ipt_TPROXY 2176 1
> iptable_tproxy 17708 1
> ip_nat 18604 1 iptable_tproxy
> ip_conntrack 49836 2 iptable_tproxy,ip_nat
> ip_tables 20096 2 ipt_TPROXY,iptable_tproxy
> ip_gre 13472 0
>
> #sysctl:
> [root@tproxy sbin]# sysctl -a | grep rp.filter
> net.ipv4.conf.gre0.arp_filter = 0
> net.ipv4.conf.gre0.rp_filter = 0
> net.ipv4.conf.eth0.arp_filter = 0
> net.ipv4.conf.eth0.rp_filter = 0
> net.ipv4.conf.default.arp_filter = 0
> net.ipv4.conf.default.rp_filter = 0
> net.ipv4.conf.all.arp_filter = 0
> net.ipv4.conf.all.rp_filter = 0
> net.ipv4.conf.lo.arp_filter = 0
> net.ipv4.conf.lo.rp_filter = 0
>
>
> many thanks & regards,
> Tino
>
> ----- Original Message -----
> From: "Steven Wilton" <swilton@q-net.net.au>
> To: "'Adrian Chadd'" <adrian@creative.net.au>; "'tino'"
> <tino.kriswanto@gmail.com>
> Cc: "'Kashif Ali Bukhari'" <kbukhari@gmail.com>;
> <devel@cacheboy.net>;
> "'chima s'" <chima.s@gmail.com>
> Sent: Wednesday, July 12, 2006 11:22 AM
> Subject: RE: [Devel] Re: [squid-users] TPROXY on squid-2.6S1
>
>
> > I've got tproxy + wccp2 working with squid 2.6. There are
> a few things
> > that
> > need to be done:
> >
> > - The kernel and iptables need to be patched with the
> tproxy patches (and
> > the tproxy include file needs to be placed in
> > /usr/include/linux/netfilter_ipv4/ip_tproxy.h or
> > include/netfilter_ipv4/ip_tproxy.h in the squid src tree).
> >
> > - The iptables rule needs to use the TPROXY target (instead of the
> > REDIRECT
> > target) to redirect the port 80 traffic to the proxy. Ie:
> > iptables -t tproxy -A PREROUTING -i eth0 -p tcp -m tcp
> --dport 80 -j
> > TPROXY
> > --on-port 80
> >
> > - The kernel must strip the GRE header from the incoming
> packets (either
> > using the ip_wccp module, or by having a GRE tunnel set up in linux
> > pointing
> > at the router (no GRE setup is required on the router)).
> >
> > - 2 wccp services must be used. We use the following wccp
> definitions:
> > wccp2_service dynamic 80
> > wccp2_service_info 80 protocol=tcp flags=src_ip_hash
> priority=240 ports=80
> > wccp2_service dynamic 90
> > wccp2_service_info 90 protocol=tcp flags=dst_ip_hash,ports_source
> > priority=240 ports=80
> >
> > It is highly recommended that the above definitions be used
> for the two
> > wccp
> > services, otherwise things will break if you have more than 1 cache
> > (specifically, you will have problems when the a web server's name
> > resolves
> > to multiple ip addresses).
> >
> > - The http port that you are redirecting to must have the
> transparent and
> > tproxy options enabled as follows (modify the port as appropriate):
> > http_port 80 transparent tproxy
> >
> > - There _must_ be a tcp_outgoing address defined. This
> will need to be
> > valid to satisfy any non-tproxied connections.
> >
> > - On the router, you need to make sure that all traffic
> going to/from the
> > customer will be processed by _both_ wccp rules. The way we have
> > implemented this is to apply wccp service 80 to all traffic
> coming in from
> > a
> > customer-facing interface, and wccp service 90 applied to
> all traffic
> > going
> > out a customer-facing interface. We have also applied the wccp
> > "exclude-in"
> > rule to all traffic coming in from the proxy-facing interface. Ie:
> >
> > interface GigabitEthernet0/3.100
> > description ADSL customers
> > encapsulation dot1Q 502
> > ip address x.x.x.x y.y.y.y
> > ip wccp 80 redirect in
> > ip wccp 90 redirect out
> >
> > interface GigabitEthernet0/3.101
> > description Sialup customers
> > encapsulation dot1Q 502
> > ip address x.x.x.x y.y.y.y
> > ip wccp 80 redirect in
> > ip wccp 90 redirect out
> >
> > interface GigabitEthernet0/3.102
> > description proxy servers
> > encapsulation dot1Q 506
> > ip address x.x.x.x y.y.y.y
> > ip wccp redirect exclude in
> >
> > - It's higly recommended to turn httpd_accel_no_pmtu_disc
> on in the squid
> > conf.
> >
> > - If you have some clients who set their proxy, it is
> recommended to use a
> > separate port in squid for transparent/tproxy requests
> compared to clients
> > with proxies set.
> >
> >
> > I'm about to post a couple of patches to the squid-dev list
> to fix 2
> > issues
> > I've found when using tproxy and squid2.6:
> > 1 - When the tproxy patch is applied, squid may be limited to 1024
> > filedescriptors (it was for me)
> > 2 - NTLM auth does not work for transparent requests
> >
> > I would imagine that these issues will be resolved in squid
> 2.6.STABLE2
> >
> > Steven
> >
> >> -----Original Message-----
> >> From: devel-bounces@cacheboy.net
> >> [mailto:devel-bounces@cacheboy.net] On Behalf Of Adrian Chadd
> >> Sent: Wednesday, 12 July 2006 10:23 AM
> >> To: tino
> >> Cc: Kashif Ali Bukhari; devel@cacheboy.net; chima s
> >> Subject: Re: [Devel] Re: [squid-users] TPROXY on squid-2.6S1
> >>
> >> On Wed, Jul 12, 2006, tino wrote:
> >> > hi, guys,
> >> > any success with tproxy at 2.6.s1 ? seems no example/howto
> >> to make it work
> >> >
> >> > you can read my last post for example config.
> >> > I am trying to use squid-2.6.S1, with wccpv2, cttproxy,
> >> kernel 2.6.15, fc4
> >> > So far no error detected, but it just not work
> >>
> >> Steven has it working with a cacheboy-squid release but I
> >> don't know what
> >> he's done with squid-2.6. I haven't had an environment
> until recently
> >> which lets me test out the squid-tproxy stuff so I can't comment
> >> either way.
> >>
> >> I do agree that it needs much, much better documentation.
> >> I'll see what I can do.
> >>
> >>
> >>
> >>
> >>
> >> Adrian
> >>
> >> _______________________________________________
> >> Devel mailing list
> >> Devel@cacheboy.net
> >> http://cacheboy.net/cgi-bin/mailman/listinfo/devel
> >>
> >> --
> >> No virus found in this incoming message.
> >> Checked by AVG Free Edition.
> >> Version: 7.1.394 / Virus Database: 268.9.10/385 - Release
> >> Date: 11/07/2006
> >>
> >>
> >
> > --
> > No virus found in this outgoing message.
> > Checked by AVG Free Edition.
> > Version: 7.1.394 / Virus Database: 268.9.10/385 - Release
> Date: 11/07/2006
> >
> >
>
>
> --
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.1.394 / Virus Database: 268.9.10/385 - Release
> Date: 11/07/2006
>
>

-- 
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.394 / Virus Database: 268.9.10/385 - Release Date: 11/07/2006