Date: Fri, 07 Sep 2007 11:58:48 -0700 From: Julian Elischer <julian@elischer.org> To: Julian Elischer <julian@elischer.org>, David Christensen <davidch@broadcom.com>, FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: FreeBSD discarding received packets > MTU Message-ID: <46E19F68.1090802@elischer.org> In-Reply-To: <20070907172216.GL977@funkthat.com> References: <46E0632D.8070200@elischer.org> <46E07E74.5020204@elischer.org> <09BFF2FA5EAB4A45B6655E151BBDD9030501D5C0@NT-IRVA-0750.brcm.ad.broadcom.com> <46E0A5DB.3080404@elischer.org> <20070907172216.GL977@funkthat.com>
next in thread | previous in thread | raw e-mail | index | archive | help
John-Mark Gurney wrote:
> Julian Elischer wrote this message on Thu, Sep 06, 2007 at 18:14 -0700:
>> David Christensen wrote:
>>>> Julian Elischer wrote:
>>>>> Ok which clever person did this again?
>>>>>
>>>>> It just broke our product.
>>>>> If it hasn't been removed from 7.0 and 6.x yet it's just
>>>> about to be...
>>>>
>>>>
>>>> on the topic above, I've found that the bce driver seems to
>>>> be throwingaway packets
>>>> larger than the mtu regardless of the rest of the system..
>>> This has been part of the driver since it's initial release,
>>> it's not an addition or change.
>>>
>>>> the following code seems to be responsible..
>>>>
>>>> in the spirit of "be generous on receive" as mentionned before,
>>>> should it be relaxed?
>>>>
>>> When this RX MTU size issue was being discussed previously I
>>> was wondering why nobody actually discussed what the drivers
>>> are doing. Both the NetXtreme (bge) and NetXtreme II (bce)
>>> controllers have a setting in the RX MAC to specify the MTU
>>> size of the medium. Anything larger than the MTU is discarded
>>> and a statistic is recorded. Support for jumbo frames is
>>> available (though not on all bge supported controllers) but the
>>> general understanding from the hardware design perspective is
>>> that the MTU is the maximum message size on the medium and that
>>> all controllers on the medium should be configured for the same
>>> message size. I've never seen an Ethernet driver that supports
>>> different MTUs for RX and TX (is there one implemented under
>>> FreeBSD?) and I would really hate to troubleshoot any higher
>>> level application problems that might occur if the RX and TX MTU
>>> were different for every system on a network.
>>>
>>>> /* Calculate and program the Ethernet MTU size. */
>>>> ether_mtu = ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN +
>>>> ifp->if_mtu +
>>>> ETHER_CRC_LEN;
>>>>
>>>> DBPRINT(sc, BCE_INFO, "%s(): setting mtu =
>>>> %d\n",__FUNCTION__, ether_mtu);
>>>>
>>>>
>>>> /*
>>>> * Program the mtu, enabling jumbo frame
>>>> * support if necessary. Also set the mbuf
>>>> * allocation count for RX frames.
>>>> */
>>>> if (ether_mtu > ETHER_MAX_LEN + ETHER_VLAN_ENCAP_LEN) {
>>>> REG_WR(sc, BCE_EMAC_RX_MTU_SIZE, ether_mtu |
>>>> BCE_EMAC_RX_MTU_SIZE_JUMBO_ENA);
>>>> sc->mbuf_alloc_size = MJUM9BYTES;
>>>> } else {
>>>> REG_WR(sc, BCE_EMAC_RX_MTU_SIZE, ether_mtu);
David, Assuming I want to just allow it to capture packets that
will fit in whatever was allocated, what would I add here?
as a guess I imagine the following?
REG_WR(sc, BCE_EMAC_RX_MTU_SIZE, MCLBYTES-(sizeof(struct l2_fhdr) + 2 + 8));
>>>> sc->mbuf_alloc_size = MCLBYTES;
>>>> }
>>>>
>>>>
>>>> could probably be relaxed in the non jumbo case to accept
>>>> packets up to
>>>> MCLBYTES in size instead of ether_mtu.
>>> Maybe MCLBYTES is good enough for you but what if someone has
>>> a different requirement? How big is big enough for everyone?
I just want the case for encapsulated packets. usually this is a small
overhead.
The BSD 'way' has always been to accept whatever we practically
can, but to only send strictly according to the spec.
(I say BSD way as in "since 1982 or so")
it's even documented somewhere.
(see Mike Karel's previous email on the topic)
>>>
>>> I suppose supporting different MTUs for RX and TX is possible,
>>> though the memory requirements are 5x larger for jumbo frames
>>> over standard frames and many users might consider this a waste
>>> of resources if they don't support jumbo frames on their
>>> network (that would be almost 4MB). Support would obviously
>>> be easier if "ifconfig" support both an RX and TX MTU size.
>>>
>>>> manually I have set the mtu to larger values which increases
>>>> ether_mtu and that seems to work..
>>>> now, Why do we need this?
>>>>
>>>> WCCP is a protocol from Cisco.
>>>> it encapsulates packets in GRE, including an extra
>>>> intermediate header.
>>>> when packets are encapsulated in GRE by cisco 6k switches
>>>> (quite a few around)
>>>> they do NOT frag the result, but instead, send an oversized packet.
>>>>
>>>> if you have a bce interface, it discards these packets which
>>>> is a drag.
>>> It could certainly be argued by some that Cisco is not standards
>>> compliant in this case for sending an oversized Ethernet frame
>>> and expecting everyone to accept it. Hardware has limitations
>>> and assuming that all Ethernet controllers can support frames
>>> greater than 1522 bytes is not reasonable. Fortunately there is
>>> a suitable workaround which is setting a larger MTU for the
>>> interface. What size do you use? How did you arrive at that
>>> value?
>> I use 1550 to make it work in the test harness.
>>
>> The trouble is that if I set the mtu to 1550, and the machine talks to
>> another
>> such machine with it's mtu also set to 1550 then they negotiate a maximum
>> sized
>> packet based on 1550, and the problem hits me again. This is a web proxy
>> and that problem occurs when there are two layers of proxy and one proxy
>> talks to another. I really just need it to to silently accept a packet some
>> 32 bytes or so larger than the stated MTU.
>
> Why not set the mtu on the net and host routes to 1500 to make it all
> work? I forget if my route mtu code is in 6.2, but it'll be in 7.0...
>
> Maybe someone will write the mtud and we can be the first OS to turn
> on jumbo frames by default.. :)
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?46E19F68.1090802>