From owner-freebsd-net@FreeBSD.ORG Sun Feb 2 01:16:51 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A4D2A453 for ; Sun, 2 Feb 2014 01:16:51 +0000 (UTC) Received: from mail-ea0-x231.google.com (mail-ea0-x231.google.com [IPv6:2a00:1450:4013:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3CAB11ED0 for ; Sun, 2 Feb 2014 01:16:51 +0000 (UTC) Received: by mail-ea0-f177.google.com with SMTP id n15so3068574ead.36 for ; Sat, 01 Feb 2014 17:16:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=jgdoD+ip3nLWHpcx2lRMhOK3owCbrehWKyxvfwhLEAs=; b=HfnjOPRr6+oRhHo+R3qtVcB8Qzm5aSynI7qwToAEC7MTFJYMRq1p2B5yMU3T3qsMW7 cUMxsbKVOC5SnQUggkgA2tu5T6OWKHAwnnvQRiZfdnxAcTQCV7NLVjf5Zktgz3ADGNho wpdoejUgQNuOuYggUGsNkM2SeqjIlHCt11D9hqSXUTCYoojjyyoMnOyaLIfI04aFtsoZ p7cx/RktQ8cRPj9KSsnA56c00cb2N6byrMB2bG0rZDCFykqdC/5UOCNDVflpQl4jLB1U vHo9K386cU39QLOF0LWqG4bA8GhNZqbJZAIW8xnutcR6TZj1y6geYtg5t4CEgfYYjx1k rR0g== MIME-Version: 1.0 X-Received: by 10.14.126.9 with SMTP id a9mr466965eei.95.1391303808483; Sat, 01 Feb 2014 17:16:48 -0800 (PST) Received: by 10.14.65.4 with HTTP; Sat, 1 Feb 2014 17:16:48 -0800 (PST) In-Reply-To: References: Date: Sat, 1 Feb 2014 17:16:48 -0800 Message-ID: Subject: Re: Errors using span interface on if_bridge(4) From: hiren panchasara To: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Feb 2014 01:16:51 -0000 I rebuilt the kernel with following change to check failure: Index: if_bridge.c =================================================================== --- if_bridge.c (revision 260789) +++ if_bridge.c (working copy) @@ -2536,6 +2536,7 @@ struct bridge_iflist *bif; struct ifnet *dst_if; struct mbuf *mc; + int error = 0; if (LIST_EMPTY(&sc->sc_spanlist)) return; @@ -2552,7 +2553,9 @@ continue; } - bridge_enqueue(sc, dst_if, mc); + error = bridge_enqueue(sc, dst_if, mc); + if (error) + printf("%s: bridge_enqueue failed\n", __func__); } } After this change and reboot, I see packets on ix3 without those bad-len errors seen before. (I also want to carefully inspect packets to make sure nothing else is going wrong) I never saw "bridge_enqueue failed" error ever. So its failing somewhere else in the stack. ix1: flags=8943 metric 0 mtu 1500 options=8403bb ether 38:ea:a7:8b:af:c4 inet6 fe80::3aea:a7ff:fe8b:afc4%ix1 prefixlen 64 scopeid 0x6 inet 10.73.149.91 netmask 0xffffff00 broadcast 10.73.149.255 nd6 options=29 media: Ethernet autoselect (10Gbase-Twinax ) status: active ix2: flags=8943 metric 0 mtu 1500 options=8407bb ether 90:e2:ba:30:73:40 inet 192.168.0.2 netmask 0xffffff00 broadcast 192.168.0.255 nd6 options=29 media: Ethernet autoselect (10Gbase-Twinax ) status: active ix3: flags=8943 metric 0 mtu 1500 options=8407bb ether 90:e2:ba:30:73:41 inet 192.168.0.3 netmask 0xffffff00 broadcast 192.168.0.255 nd6 options=29 media: Ethernet autoselect (10Gbase-Twinax ) status: active bridge0: flags=8843 metric 0 mtu 1500 ether 02:a1:25:9a:8f:00 nd6 options=9 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: ix1 flags=143 ifmaxaddr 0 port 6 priority 128 path cost 2000 member: ix2 flags=8 ifmaxaddr 0 port 7 priority 128 path cost 2000 Interestingly I still see: -bash-4.2$ sysctl -a dev.ix | grep total_pkts_rcvd dev.ix.0.mac_stats.total_pkts_rcvd: 638917 dev.ix.1.mac_stats.total_pkts_rcvd: 3587878 <--- where I send traffic via iperf dev.ix.2.mac_stats.total_pkts_rcvd: 1 dev.ix.3.mac_stats.total_pkts_rcvd: 3555298 <--- where I snoop traffic. -bash-4.2$ sysctl -a | grep mac_stats.checksum_errs dev.ix.0.mac_stats.checksum_errs: 0 dev.ix.1.mac_stats.checksum_errs: 0 dev.ix.2.mac_stats.checksum_errs: 0 dev.ix.3.mac_stats.checksum_errs: 3371119 <-- most of them fail checksum I also noticed: dev.ix.3.queue4.lro_queued: 50223 dev.ix.3.queue4.lro_flushed: 42632 dev.ix.3.queue5.lro_queued: 25356 dev.ix.3.queue5.lro_flushed: 25249 I disabled tso and lro both and I stopped seeing these number increase. But the general behavior remained the same. cheers, Hiren