Date: Mon, 23 May 2011 12:16:39 +0200 From: Attila Nagy <bra@fsn.hu> To: pyunyh@gmail.com Cc: freebsd-net@freebsd.org, rmacklem@uoguelph.ca, sthaug@nethelp.no, Ronald Klop <ronald-freebsd8@klop.yi.org> Subject: Re: TSO ethernet frame errors on 8-STABLE, was: bogus 0 len IP packet Message-ID: <4DDA3407.8020200@fsn.hu> In-Reply-To: <20110207002235.GA1244@michelle.cdnetworks.com> References: <op.vpekz9uz8527sy@212-123-145-58.ip.telfort.nl> <708793006.292748.1295186099577.JavaMail.root@erie.cs.uoguelph.ca> <20110117005524.GA1305@michelle.cdnetworks.com> <20110118.093804.74673434.sthaug@nethelp.no> <op.vpokw9c58527sy@pinky> <op.vqh69nx08527sy@212-123-145-58.ip.telfort.nl> <20110207002235.GA1244@michelle.cdnetworks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format. --------------020101040807050706060604 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi, On 02/07/11 01:22, Pyun YongHyeon wrote: > On Sun, Feb 06, 2011 at 11:54:49PM +0100, Ronald Klop wrote: >> On Sat, 22 Jan 2011 00:01:47 +0100, Ronald Klop >> <ronald-freebsd8@klop.yi.org> wrote: >> >>> On Tue, 18 Jan 2011 09:38:04 +0100,<sthaug@nethelp.no> wrote: >>> >>>>>> So, does anyone have an idea why the IP length field would be set to >>>>> 0 >>>>>> for these TCP/IP packets? >>>>>> >>>>>> Here's some info from Ronald w.r.t. his hardware. (All I can think >>>>> of is >>>>>> that he could try disabling TSO, etc?) >>>>>> >>>>>> Thanks in advance for any help with this, rick >>>>>> >>>>> It seems that issue came from TSO. Driver will set ip_len and >>>>> ip_sum field to 0 before passing the TCP segment to controller. >>>>> The failed length were 4446, 5858, 3034 and 4310 and the total >>>>> number of such frames are more than 35k within 90 seconds. Since >>>>> failed length 4310 is continuously repeated I guess there is edge >>>>> case where em(4) didn't free failed TCP segment for TSO. >>>>> I remember there was commit to HEAD(r217295) which could be related >>>>> with this issue. >>>> I'm seeing the same problem with Broadcom NetXtreme (bce) cards: >>>> >>>> bce0@pci0:3:0:0: class=0x020000 card=0x03421014 chip=0x164c14e4 >>>> rev=0x12 hdr=0x00 >>>> vendor = 'Broadcom Corporation' >>>> device = 'Broadcom NetXtreme II Gigabit Ethernet Adapter >>>> (BCM5708)' >>>> class = network >>>> subclass = ethernet >>>> >>>> This is with 8.2-PRERELEASE. Turning off TSO (ifconfig bce0 -tso) >>>> removes the problem. >>>> >>>> Steinar Haug, Nethelp consulting, sthaug@nethelp.no >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >>> I tried -tso and -txcsum in various combinations, but it didn't solve >>> the problem. I wil look for another brand of network card to try. But >>> this has to wait till monday when I'm at the office again. >> I also used another network card (rl0) and it has the same problem with >> NFS. I'm going to change some network cables to see if that helps. I have >> some hints that there might be something wrong with that. >> > Hmm, given that rl(4) also shows the issue it seems the issue could > be in TCP/IP stack, not in driver side. rl(4) is dumb device so > network stack should do segmentation and checksum computation. > I highly doubt the issue came from faulty cable since other users > also reported the same issue. > Unfortunately I have no clue yet and I was not able to reproduce it > on my box. I vaguely guess some code in kernel changed the ip_len > to 0 in the middle of transmission. Rick's captured traffic looks > normal except 0 ip_len given that controller is computing checksum > on the fly. If mbuf chain was corrupted(e.g. m_len == 0) driver > would have failed to send those frames. I can see something similar. Attached are two pcaps, with TSO enabled/disabled (net.inet.tcp.tso). NIC is: bce0: <HP NC373i Multifunction Gigabit Server Adapter (B2)> mem 0xfa000000-0xfbffffff irq 16 at device 0.0 on pci7 miibus0: <MII bus> on bce0 brgphy0: <BCM5708S 1000/2500BaseSX PHY> PHY 2 on miibus0 brgphy0: 1000baseSX-FDX, 2500baseSX-FDX, auto bce0: Ethernet address: 00:1f:29:cc:97:62 bce0: [ITHREAD] In a HP BL460c. The OS is a 8-STABLE/amd64, compiled two days ago. --------------020101040807050706060604 Content-Type: application/octet-stream; name="tso-disabled" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="tso-disabled" 1MOyoQIABAAAAAAAAAAAAP//AAABAAAA6DLaTYsIAQBKAAAASgAAAAAfKcyXYgAdcNGtqwgA RQAAPMxnQABABmTlwKhEG8CoRAMjz5xIzgr0SAAAAACgAhbQEN0AAAIEBbQEAggKXFA4LwAA AAABAwMA6DLaTa4IAQBKAAAASgAAAAAdcNGtqwAfKcyXYggARQAAPD+UQABABvG4wKhEA8Co RBucSCPPtWwMb84K9EmgEv//oPAAAAIEBbQBAwMDBAIICpq0KhhcUDgv6DLaTbgKAQBCAAAA QgAAAAAfKcyXYgAdcNGtqwgARQAANMxoQABABmTswKhEG8CoRAMjz5xIzgr0SbVsDHCAEBbQ uOgAAAEBCApcUDgvmrQqGOgy2k1GDAEAewAAAHsAAAAAHynMl2IAHXDRrasIAEUAAG3MaUAA QAZkssCoRBvAqEQDI8+cSM4K9Em1bAxwgBgW0ECpAAABAQgKXFA4L5q0KhhHRVQgLyBIVFRQ LzEuMQ0KQ29ubmVjdGlvbjogQ2xvc2UNCkhvc3Q6IDE5Mi4xNjguNjguMw0KDQroMtpN04cB AOoFAADqBQAAAB1w0a2rAB8pzJdiCABFAAXcP5tAAEAG7BHAqEQDwKhEG5xII8+1bAxwzgr0 goAQIIYIYgAAAQEICpq0KjlcUDgvSFRUUC8xLjEgNDA0IE5vdCBGb3VuZA0KQ29udGVudC1U eXBlOiB0ZXh0L2h0bWw7IGNoYXJzZXQ9aXNvLTg4NTktMQ0KQ2FjaGUtQ29udHJvbDogbXVz dC1yZXZhbGlkYXRlLG5vLWNhY2hlLG5vLXN0b3JlDQpDb250ZW50LUxlbmd0aDogMTM2NQ0K Q29ubmVjdGlvbjogY2xvc2UNClNlcnZlcjogSmV0dHkoNi4xLjIxKQ0KDQo8aHRtbD4KPGhl YWQ+CjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIgY29udGVudD0idGV4dC9odG1s OyBjaGFyc2V0PUlTTy04ODU5LTEiLz4KPHRpdGxlPkVycm9yIDQwNCBOT1RfRk9VTkQ8L3Rp dGxlPgo8L2hlYWQ+Cjxib2R5PjxoMj5IVFRQIEVSUk9SIDQwNDwvaDI+CjxwPlByb2JsZW0g YWNjZXNzaW5nIC8uIFJlYXNvbjoKPHByZT4gICAgTk9UX0ZPVU5EPC9wcmU+PC9wPjxociAv PjxpPjxzbWFsbD5Qb3dlcmVkIGJ5IEpldHR5Oi8vPC9zbWFsbD48L2k+PGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICDoMtpN4YcBAKgAAACoAAAAAB1w0a2rAB8pzJdiCABFAACaP5xAAEAG8VLA qEQDwKhEG5xII8+1bBIYzgr0goAYIIaMqgAAAQEICpq0KjlcUDgvICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgCjxici8+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgCgo8L2JvZHk+CjwvaHRtbD4K6DLaTQiIAQBCAAAAQgAAAAAdcNGt qwAfKcyXYggARQAAND+dQABABvG3wKhEA8CoRBucSCPPtWwSfs4K9IKAESCGqMkAAAEBCAqa tCo5XFA4L+gy2k1riQEAQgAAAEIAAAAAHynMl2IAHXDRrasIAEUAADTMakAAQAZk6sCoRBvA qEQDI8+cSM4K9IK1bBIYgBAh8KfDAAABAQgKXFA4Mpq0KjnoMtpNn4kBAEIAAABCAAAAAB8p zJdiAB1w0a2rCABFAAA0zGtAAEAGZOnAqEQbwKhEAyPPnEjOCvSCtWwSfoAQIfCnXQAAAQEI ClxQODKatCo56DLaTSWKAQBCAAAAQgAAAAAfKcyXYgAdcNGtqwgARQAANMxsQABABmTowKhE G8CoRAMjz5xIzgr0grVsEn+AESHwp1sAAAEBCApcUDgymrQqOegy2k1IigEAQgAAAEIAAAAA HXDRrasAHynMl2IIAEUAADQ/nkAAQAbxtsCoRAPAqEQbnEgjz7VsEn/OCvSDgBAghajGAAAB AQgKmrQqOVxQODI= --------------020101040807050706060604 Content-Type: application/octet-stream; name="tso-enabled" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="tso-enabled" 1MOyoQIABAAAAAAAAAAAAP//AAABAAAASTHaTWdVBwBKAAAASgAAAAAfKcyXYgAVLFEmAAgA RQAAPJqVQAA6Bjy3wKikG8CoRAP2sZxIsuVNXQAAAACgAhbQNoQAAAIEBbQEAggKJj3XyAAA AAABAwMASTHaTYNVBwBKAAAASgAAAAAADAesdAAfKcyXYggARQAAPMqIQABABgbEwKhEA8Co pBucSPaxKXzN4LLlTV6gEv//okMAAAIEBbQBAwMDBAIICt8e1IAmPdfISTHaTT1aBwBCAAAA QgAAAAAfKcyXYgAVLFEmAAgARQAANJqWQAA6Bjy+wKikG8CoRAP2sZxIsuVNXil8zeGAEBbQ ujsAAAEBCAomPdfI3x7UgEkx2k3BWgcAewAAAHsAAAAAHynMl2IAFSxRJgAIAEUAAG2al0AA OgY8hMCopBvAqEQD9rGcSLLlTV4pfM3hgBgW0EH8AAABAQgKJj3XyN8e1IBHRVQgLyBIVFRQ LzEuMQ0KQ29ubmVjdGlvbjogQ2xvc2UNCkhvc3Q6IDE5Mi4xNjguNjguMw0KDQpJMdpN/F0H AFAGAABQBgAAAAAMB6x0AB8pzJdiCABFAAAAyolAAEAGAADAqEQDwKikG5xI9rEpfM3hsuVN l4AYIIbtTQAAAQEICt8e1IImPdfISFRUUC8xLjEgNDA0IE5vdCBGb3VuZA0KQ29udGVudC1U eXBlOiB0ZXh0L2h0bWw7IGNoYXJzZXQ9aXNvLTg4NTktMQ0KQ2FjaGUtQ29udHJvbDogbXVz dC1yZXZhbGlkYXRlLG5vLWNhY2hlLG5vLXN0b3JlDQpDb250ZW50LUxlbmd0aDogMTM2NQ0K Q29ubmVjdGlvbjogY2xvc2UNClNlcnZlcjogSmV0dHkoNi4xLjIxKQ0KDQo8aHRtbD4KPGhl YWQ+CjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIgY29udGVudD0idGV4dC9odG1s OyBjaGFyc2V0PUlTTy04ODU5LTEiLz4KPHRpdGxlPkVycm9yIDQwNCBOT1RfRk9VTkQ8L3Rp dGxlPgo8L2hlYWQ+Cjxib2R5PjxoMj5IVFRQIEVSUk9SIDQwNDwvaDI+CjxwPlByb2JsZW0g YWNjZXNzaW5nIC8uIFJlYXNvbjoKPHByZT4gICAgTk9UX0ZPVU5EPC9wcmU+PC9wPjxociAv PjxpPjxzbWFsbD5Qb3dlcmVkIGJ5IEpldHR5Oi8vPC9zbWFsbD48L2k+PGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPGJyLz4gICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKCjwvYm9keT4KPC9o dG1sPgpJMdpNJF4HAEIAAABCAAAAAAAMB6x0AB8pzJdiCABFAAA0yopAAEAGBsrAqEQDwKik G5xI9rEpfNPvsuVNl4ARIIaqOgAAAQEICt8e1IMmPdfISTHaTW1hBwBCAAAAQgAAAAAfKcyX YgAVLFEmAAgARQAANJqYQAA6Bjy8wKikG8CoRAP2sZxIsuVNlyl804mAECHwqTgAAAEBCAom PdfI3x7Ugkkx2k2WYQcAQgAAAEIAAAAAHynMl2IAFSxRJgAIAEUAADSamUAAOgY8u8CopBvA qEQD9rGcSLLlTZcpfNPvgBAh8KjSAAABAQgKJj3XyN8e1IJJMdpNTmIHAEIAAABCAAAAAB8p zJdiABUsUSYACABFAAA0mppAADoGPLrAqKQbwKhEA/axnEiy5U2XKXzT8IARIfCozwAAAQEI CiY918jfHtSDSTHaTV5iBwBCAAAAQgAAAAAADAesdAAfKcyXYggARQAANMqMQABABgbIwKhE A8CopBucSPaxKXzT8LLlTZiAECCFqjkAAAEBCArfHtSEJj3XyA== --------------020101040807050706060604--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4DDA3407.8020200>