From owner-freebsd-bugs@FreeBSD.ORG Mon Sep 22 10:20:24 2003 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0936816A4BF for ; Mon, 22 Sep 2003 10:20:24 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7B28943FFB for ; Mon, 22 Sep 2003 10:20:11 -0700 (PDT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h8MHKBFY097533 for ; Mon, 22 Sep 2003 10:20:11 -0700 (PDT) (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h8MHKB4b097532; Mon, 22 Sep 2003 10:20:11 -0700 (PDT) (envelope-from gnats) Resent-Date: Mon, 22 Sep 2003 10:20:11 -0700 (PDT) Resent-Message-Id: <200309221720.h8MHKB4b097532@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, pak@cns.utoronto.ca Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 27BB716A4B3 for ; Mon, 22 Sep 2003 10:18:26 -0700 (PDT) Received: from rodent.utcs.utoronto.ca (rodent.utcs.utoronto.ca [128.100.102.5]) by mx1.FreeBSD.org (Postfix) with SMTP id 1833E43FDD for ; Mon, 22 Sep 2003 10:18:25 -0700 (PDT) (envelope-from pkern@cns.utoronto.ca) Received: by rodent.utcs.utoronto.ca id <1650488>; Mon, 22 Sep 2003 13:18:12 -0400 Message-Id: <03Sep22.131812edt.1650488@rodent.utcs.utoronto.ca> Date: Mon, 22 Sep 2003 13:18:12 -0400 From: pak@cns.utoronto.ca To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 cc: pak@cns.utoronto.ca Subject: kern/57100: disable hardware checksums when using bridge(4). X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: pak@cns.utoronto.ca List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Sep 2003 17:20:24 -0000 >Number: 57100 >Category: kern >Synopsis: disable hardware checksums when using bridge(4). >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: change-request >Submitter-Id: current-users >Arrival-Date: Mon Sep 22 10:20:10 PDT 2003 >Closed-Date: >Last-Modified: >Originator: pak >Release: FreeBSD 4.8-RELEASE i386 >Organization: Computing and Network Services, U Toronto >Environment: System: FreeBSD host1 4.8-RELEASE FreeBSD 4.8-RELEASE #2: Tue Sep 16 14:18:09 EDT 2003 pak@host1:/usr/src/sys/compile/BOX i386 - dmesg excerpts: ... xl0: <3Com 3c905B-TX Fast Etherlink XL> port [...] ... em0: port [...] ... BRIDGE 020214 loaded ... host1# ifconfig xl0: flags=8843 mtu 1500 options=3 inet 300.2.2.2 netmask 0xffffff00 broadcast 300.2.2.255 [...] em0: flags=8843 mtu 1500 options=3 inet 10.1.1.1 netmask 0xfffffff8 broadcast 10.1.1.7 [...] host1# sysctl -w net.link.ether.bridge=1 host1# sysctl -w net.link.ether.bridge_cfg="xl0,em0" - xl0 is directly connected to the "outside world." - xl0 is configured with a proper IP address (eg. 300.2.2.2 ). - em0 is configured with a non-routable RFC-1918 address (say 10.1.1.1). - em0 is connected to a switch along with another system called inner1 so that traffic to and/or from inner1 is being bridged by host1. - a rough diagram: :-----------: { the } : host1 : | {outside} ----- xl0 em0 ----+ { world } : : | :-----------: | :--------: +--- : inner1 : | :--------: >Description: When bridging is enabled, hardware checksumming by the NICs corrupts the checksums for packets which are generated by the bridging system and intended for systems "on the inside". In the example, with bridging enabled, inner1 is able to talk to the outside world but it's not able to talk to host1 via host1's "proper" address. IP packets generated by host1 and intended for inner1 have incorrect checksums. Disabling hardware checksums during bridging appears to fix this. >How-To-Repeat: >Fix: Included below is a patch to disable (restore) the use of hardware checksums on bridging NICs when bridging is enabled (disabled). The same sort of manipulation of 'hwassist' can be found in /sys/netgraph/ng_ether.c. Hope this helps. pak. *** net/bridge.c 2003/09/22 15:09:35 1.1 --- net/bridge.c 2003/09/22 14:51:23 *************** *** 314,319 **** --- 314,320 ---- if ( b->flags & IFF_BDG_PROMISC ) { s = splimp(); + ifp->if_hwassist = b->hwassist; /* restore hwassist. */ ifpromisc(ifp, 0); splx(s); b->flags &= ~(IFF_BDG_PROMISC|IFF_MUTE) ; *************** *** 361,366 **** --- 362,369 ---- if ( !(b->flags & IFF_BDG_PROMISC) ) { int ret ; s = splimp(); + b->hwassist = ifp->if_hwassist; /* save hwassist. */ + ifp->if_hwassist = 0; /* disable hwassist. */ ret = ifpromisc(ifp, 1); splx(s); b->flags |= IFF_BDG_PROMISC ; *** net/bridge.h 2003/09/22 14:52:05 1.1 --- net/bridge.h 2003/09/22 14:53:12 *************** *** 47,52 **** --- 47,53 ---- #define IFF_BDG_PROMISC 0x0001 /* set promisc mode on this if. */ #define IFF_MUTE 0x0002 /* mute this if for bridging. */ #define IFF_USED 0x0004 /* use this if for bridging. */ + u_long hwassist; /* hardware checksum capabilities */ struct cluster_softc *cluster; } ; >Release-Note: >Audit-Trail: >Unformatted: