From owner-freebsd-net@FreeBSD.ORG Wed Nov 19 00:16:28 2008 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D3581065672 for ; Wed, 19 Nov 2008 00:16:28 +0000 (UTC) (envelope-from security@jim-liesl.org) Received: from smtp1.mc.surewest.net (qsmtp.mc.surewest.net [66.60.130.145]) by mx1.freebsd.org (Postfix) with SMTP id 47A058FC25 for ; Wed, 19 Nov 2008 00:16:28 +0000 (UTC) (envelope-from security@jim-liesl.org) Received: (qmail 2250 invoked from network); 18 Nov 2008 15:59:18 -0800 Received: by simscan 1.1.0 ppid: 2247, pid: 2248, t: 0.0663s scanners: regex: 1.1.0 attach: 1.1.0 Received: from unknown (HELO smtp.jim-liesl.org) (66.60.173.44) by smtp1 with SMTP; 18 Nov 2008 15:59:18 -0800 Received: from smtp.jim-liesl.org (localhost.static.surewest.net [127.0.0.1]) by smtp.jim-liesl.org (Postfix) with ESMTP id 423E55DBB; Tue, 18 Nov 2008 16:16:25 -0800 (PST) Received: from [IPv6:::1] (daemon.static.surewest.net [192.168.1.15]) by smtp.jim-liesl.org (Postfix) with ESMTP id A296F5DB8; Tue, 18 Nov 2008 16:16:19 -0800 (PST) Message-ID: <49235AD3.1090702@jim-liesl.org> Date: Tue, 18 Nov 2008 16:16:19 -0800 From: security User-Agent: Thunderbird 2.0.0.17 (Windows/20080914) MIME-Version: 1.0 To: Jung-uk Kim References: <492302E2.4040907@jim-liesl.org> <200811181344.20092.jkim@FreeBSD.org> In-Reply-To: <200811181344.20092.jkim@FreeBSD.org> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV using ClamSMTP Cc: freebsd-net@FreeBSD.org Subject: Re: bce discard frame w/o leading ethernet header and polling (broken?) 7.1-beta2 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Nov 2008 00:16:28 -0000 Jung-uk Kim wrote: > On Tuesday 18 November 2008 01:01 pm, security wrote: > >> I'm building a WAN emulation box based on 7.1-beta2-ipfw and >> dummynet. The config is basically a router-on-a-stick. The server >> (FBSD rtr) has two nics which connect to two different switches, >> but both switch ports are in the same untagged interconnected vlan. >> All the other test boxes in the network are also in the same vlan, >> but broken into different nets. The different nets are spread >> across the two nics as primary and alias ip address in different >> nets. I've been getting "bursts" (maybe 8-20 at a time) of the >> discard frame messages. Mostly on bce1 but I've seen them on bce0 >> also. bce1 is probably the busier of the 2 currently. As a shot in >> the dark, I disabled polling system wide and the messages seemed to >> have stopped. >> > > Please try the latest driver from RELENG_7. The following commit > seems interesting: > > http://svn.freebsd.org/viewvc/base?view=revision&revision=184826 > > Jung-uk Kim > > >> thanks >> jim >> >> >> kernel: bce1: discard frame w/o leading ethernet header (len >> 4294967292 pkt len 4294967292) >> >> ipfw/dummynet/pipes are being used to rate limit traffic by src/dst >> ip address. >> >> The FreeBSD box uses the broadcom bcm5706s gigE interfaces. >> class=0x020000 card=0x310c103c chip=0x16aa14e4 rev=0x02 hdr=0x00. >> Based on some readings, I've got the following mods in place: >> i386 sources running on a 2 x dual core athalon64 cpus, 4 cores >> active. 8gig of mem available, but only 4 in use >> kernel >> i486 and i586 commented out >> nfs options commented out >> fbsd 4 and 5 commented out >> hz=1000 >> ipfirewall >> ipfirewall_default_to accept >> dummynet >> eisa commented out as well as the older nics >> >> sysctl settings >> kern.polling.enable=1 (setting this to 0 seems to fix the problem, >> but leaves the cpu's busier) >> kern.ipc.maxsockbuf=16777216 (not sure this helps much in the case >> of a rtr) net.inet.ip.forwarding=1 >> net.inet..tcp.sendbuf_auto=1 >> net.inet..tcp.sendbuf_max=16777216 >> net.inet..tcp.recvbuf_auto=1 >> net.inet..tcp.recvbuf_max=16777216 >> net.inet.tcp.rfc1323=1 >> net.link.ether.inet.log_arp_wrong_iface=0 (to suppress the arp >> messages) >> I'm running the new driver now. It's logging Ierrs on the of "netstat -I bcen -a" @ about 1-2 per second. I don't *think* it was doing that under the original driver, but I couldn't swear to it, and it would be difficult to revert right now to check. The errors seem unrelated to traffic generated by ping repeat count or payload size. Might these be the ARP on the wrong iface errors I'm suppressing ? Ping isn't complaining about any dropped packets and I'm not seeing any drops or collision stats. jim