From owner-freebsd-net@FreeBSD.ORG Thu Sep 6 23:39:07 2007 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B01B16A41A for ; Thu, 6 Sep 2007 23:39:07 +0000 (UTC) (envelope-from davidch@broadcom.com) Received: from mms2.broadcom.com (mms2.broadcom.com [216.31.210.18]) by mx1.freebsd.org (Postfix) with ESMTP id EA7B613C46B for ; Thu, 6 Sep 2007 23:39:06 +0000 (UTC) (envelope-from davidch@broadcom.com) Received: from [10.10.64.154] by mms2.broadcom.com with ESMTP (Broadcom SMTP Relay (Email Firewall v6.3.1)); Thu, 06 Sep 2007 16:38:51 -0700 X-Server-Uuid: A6C4E0AE-A7F0-449F-BAE7-7FA0D737AC76 Received: by mail-irva-10.broadcom.com (Postfix, from userid 47) id 4AA532AF; Thu, 6 Sep 2007 16:38:51 -0700 (PDT) Received: from mail-irva-8.broadcom.com (mail-irva-8 [10.10.64.221]) by mail-irva-10.broadcom.com (Postfix) with ESMTP id 367922AE; Thu, 6 Sep 2007 16:38:51 -0700 (PDT) Received: from mail-irva-12.broadcom.com (mail-irva-12.broadcom.com [10.10.64.146]) by mail-irva-8.broadcom.com (MOS 3.7.5a-GA) with ESMTP id FQP88589; Thu, 6 Sep 2007 16:38:35 -0700 (PDT) Received: from NT-IRVA-0750.brcm.ad.broadcom.com ( nt-irva-0750.brcm.ad.broadcom.com [10.8.194.64]) by mail-irva-12.broadcom.com (Postfix) with ESMTP id 20D0769CB1; Thu, 6 Sep 2007 16:38:35 -0700 (PDT) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Date: Thu, 6 Sep 2007 16:38:35 -0700 Message-ID: <09BFF2FA5EAB4A45B6655E151BBDD9030501D5C0@NT-IRVA-0750.brcm.ad.broadcom.com> In-Reply-To: <46E07E74.5020204@elischer.org> Thread-Topic: FreeBSD discarding received packets > MTU Thread-Index: Acfw1QLS5vDnuUSSTii9qkh3Gq0u0QAAluow References: <46E0632D.8070200@elischer.org> <46E07E74.5020204@elischer.org> From: "David Christensen" To: "Julian Elischer" , "FreeBSD Net" X-WSS-ID: 6AFE50012M81035300-01-01 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Cc: Subject: RE: FreeBSD discarding received packets > MTU X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Sep 2007 23:39:07 -0000 > Julian Elischer wrote: > > Ok which clever person did this again? > >=20 > > It just broke our product. > > If it hasn't been removed from 7.0 and 6.x yet it's just=20 > about to be... >=20 >=20 > on the topic above, I've found that the bce driver seems to=20 > be throwingaway packets > larger than the mtu regardless of the rest of the system.. This has been part of the driver since it's initial release, it's not an addition or change. >=20 > the following code seems to be responsible.. >=20 > in the spirit of "be generous on receive" as mentionned before, > should it be relaxed? >=20 When this RX MTU size issue was being discussed previously I was wondering why nobody actually discussed what the drivers=20 are doing. Both the NetXtreme (bge) and NetXtreme II (bce) controllers have a setting in the RX MAC to specify the MTU size of the medium. Anything larger than the MTU is discarded and a statistic is recorded. Support for jumbo frames is available (though not on all bge supported controllers) but the general understanding from the hardware design perspective is that the MTU is the maximum message size on the medium and that all controllers on the medium should be configured for the same message size. I've never seen an Ethernet driver that supports different MTUs for RX and TX (is there one implemented under=20 FreeBSD?) and I would really hate to troubleshoot any higher=20 level application problems that might occur if the RX and TX MTU=20 were different for every system on a network. >=20 >=20 > /* Calculate and program the Ethernet MTU size. */ > ether_mtu =3D ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN +=20 > ifp->if_mtu + > ETHER_CRC_LEN; >=20 > DBPRINT(sc, BCE_INFO, "%s(): setting mtu =3D=20 > %d\n",__FUNCTION__, ether_mtu); >=20 >=20 > /* > * Program the mtu, enabling jumbo frame > * support if necessary. Also set the mbuf > * allocation count for RX frames. > */ > if (ether_mtu > ETHER_MAX_LEN + ETHER_VLAN_ENCAP_LEN) { > REG_WR(sc, BCE_EMAC_RX_MTU_SIZE, ether_mtu | > BCE_EMAC_RX_MTU_SIZE_JUMBO_ENA); > sc->mbuf_alloc_size =3D MJUM9BYTES; > } else { > REG_WR(sc, BCE_EMAC_RX_MTU_SIZE, ether_mtu); > sc->mbuf_alloc_size =3D MCLBYTES; > } >=20 >=20 > could probably be relaxed in the non jumbo case to accept=20 > packets up to=20 > MCLBYTES in size instead of ether_mtu. Maybe MCLBYTES is good enough for you but what if someone has a different requirement? How big is big enough for everyone? I suppose supporting different MTUs for RX and TX is possible, though the memory requirements are 5x larger for jumbo frames over standard frames and many users might consider this a waste of resources if they don't support jumbo frames on their=20 network (that would be almost 4MB). Support would obviously be easier if "ifconfig" support both an RX and TX MTU size. >=20 > manually I have set the mtu to larger values which increases > ether_mtu and that seems to work.. > now, Why do we need this? >=20 > WCCP is a protocol from Cisco. > it encapsulates packets in GRE, including an extra=20 > intermediate header. > when packets are encapsulated in GRE by cisco 6k switches=20 > (quite a few around) > they do NOT frag the result, but instead, send an oversized packet. >=20 > if you have a bce interface, it discards these packets which=20 > is a drag. It could certainly be argued by some that Cisco is not standards compliant in this case for sending an oversized Ethernet frame and expecting everyone to accept it. Hardware has limitations and assuming that all Ethernet controllers can support frames greater than 1522 bytes is not reasonable. Fortunately there is a suitable workaround which is setting a larger MTU for the=20 interface. What size do you use? How did you arrive at that value? Dave