From owner-freebsd-net@FreeBSD.ORG Tue Feb 14 15:54:39 2006 Return-Path: X-Original-To: freebsd-net@FreeBSD.org Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E0EB916A449 for ; Tue, 14 Feb 2006 15:54:38 +0000 (GMT) (envelope-from bedinelli@madhaus.cns.utoronto.ca) Received: from madhaus.cns.utoronto.ca (madhaus.cns.utoronto.ca [128.100.103.10]) by mx1.FreeBSD.org (Postfix) with SMTP id 2E15243D4C for ; Tue, 14 Feb 2006 15:54:37 +0000 (GMT) (envelope-from bedinelli@madhaus.cns.utoronto.ca) Received: (qmail 11634 invoked by uid 31014); 14 Feb 2006 15:54:36 -0000 Received: from [128.100.103.148] (HELO [128.100.103.148]) (128.100.103.148) by madhaus.cns.utoronto.ca (qpsmtpd/0.30) with ESMTP; Tue, 14 Feb 2006 10:54:36 -0500 In-Reply-To: <20060214074606.GH86448@FreeBSD.org> References: <7bb8f24157080b6aaacb897a99259df9@madhaus.cns.utoronto.ca> <20060214074606.GH86448@FreeBSD.org> Mime-Version: 1.0 (Apple Message framework v623) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <858977c4a80306329f68021200154126@madhaus.cns.utoronto.ca> Content-Transfer-Encoding: 7bit From: Marcos Bedinelli Date: Tue, 14 Feb 2006 10:54:34 -0500 To: Gleb Smirnoff X-Mailer: Apple Mail (2.623) Cc: freebsd-net@FreeBSD.org, andre@FreeBSD.org Subject: Re: Network performance in a dual CPU system X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Feb 2006 15:54:40 -0000 Gleb, thanks again for looking into this and for your suggestions. Unfortunately Alpha/Beta/release candidate/pre-release/test versions of software is a "no go" on that machine. Our short term solution will be to upgrade the CPU to a faster model. After that, I will be able to assemble a dual processor system from spare parts and also play with fastforwarding. I will be glad to post the results of my tests to the mailing list, but it may take awhile. Cheers! On 14-Feb-06, at 9:52, Gleb Smirnoff wrote: > On Tue, Feb 14, 2006 at 09:27:46AM -0500, Marcos Bedinelli wrote: > M> >M> I tried the 'sysctl net.inet.ip.fastforwarding=1' yesterday > morning > M> >M> and, as soon as I entered the command, the machine stopped > M> >forwarding > M> >M> packets. Unfortunately I was unable to spend time > troubleshooting > M> >the > M> >M> problem because the machine is in production and can't be down > for > M> >not > M> >M> even a couple of minutes. > M> > > M> >This is strange. Can you please show output of 'ifconfig'? Are you > M> >using > M> >any packet filters or other additional processing of forwarded > packets? > M> >Did the router generated any ICMP errors or just silently dropped > M> >packets? > M> > M> > M> The machine is running Quagga (OSPF and BGP). OSPF runs on bge0. BGP > M> runs on bge1. It also runs IPFW and basic SNMP. > M> > M> I suspect the problem I experienced yesterday is related to the IPFW > M> ruleset. Although it's mainly used for blocking, there are two rules > M> that forward packets to another machine: > M> > M> [...] > M> 07700 fwd a.b.c.d ip from w.x.y.z/16 to any in recv bge0 > M> [...] > M> 15300 fwd a.b.c.d ip from any to any out xmit disc0 > M> > M> > M> Rule 7700 is doing policy-based routing: if the source IP belongs to > M> network w.x.y.z/16, the packet is forwarded to host a.b.c.d > M> > M> I am suspicious about rule 15300. Virtual interface disc0 is our > M> default route. In other words, the routing table is updated by both > M> OSPF and BGP. Packets that don't match a specific entry on the > routing > M> table are sent to disc0. Rule 15300 will catch those packets and > send > M> them to host a.b.c.d as well. > > I understood the problem. The fastforwarding code refuses to forward > packet if it is routed to interface w/o IFF_DRV_RUNNING flag. The disc0 > interface in 6.0-RELEASE doesn't have IFF_DRV_RUNNING flag. > > That's why I added this flag to disc(4) in 6.0-STABLE, to satisfy fast > forwarding: > > > http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/net/if_disc.c.diff? > r1=1.48&r2=1.48.2.1 > > Actually, it is incorrect that upper netowork layer looks into > if_drv_flags. > > You can either apply the above patch, or upgrade your system to > 6.1-BETA. As I > said before bge(4) driver has undergone some improvements, and I'll > appreciate > if you try to upgrade and provide any feedback. > > -- > Totus tuus, Glebius. > GLEBIUS-RIPN GLEB-RIPE -- Marcos