From owner-freebsd-net@FreeBSD.ORG Sun Dec 15 00:06:38 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 23D44141 for ; Sun, 15 Dec 2013 00:06:38 +0000 (UTC) Received: from mail-qa0-x22f.google.com (mail-qa0-x22f.google.com [IPv6:2607:f8b0:400d:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D74121871 for ; Sun, 15 Dec 2013 00:06:37 +0000 (UTC) Received: by mail-qa0-f47.google.com with SMTP id w5so591697qac.13 for ; Sat, 14 Dec 2013 16:06:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=+AHhCp+HwyLXbti7/kRtFYQ3XcGxQ5UvDrYbss+RBpM=; b=wVIb5idsOuTtCulshrwo8DzxjhlYT1lrPCmWkwWsGL5G5eYXlMy71AJHXQBBBSEy8C 3WsDmnuzz2UYSl4B+IZCczLFyOT42/DL0lw+m6FHOCfUsA67ZKcZL7JgcNHrhcd9iDzf eFOXacg3rO3pXmsS6Y7Nc9wgfFyorH4SWpUbD7FqwPkCVu5HkFvi2lpi0cqcgB+vxoP6 Oeb3pscfuaWrWs/8sAXhx+cKNNmPCNd2zWMldTqdEVWC9Am7CIQowrTQfXL7zfjIqWs0 3F83+U83P2t0mOdiz5PHIItarDangkJRTLQi5S212jJB1cKhZErvIyceGwiHDZ0L91zN HfXg== MIME-Version: 1.0 X-Received: by 10.49.116.141 with SMTP id jw13mr19229799qeb.2.1387065997119; Sat, 14 Dec 2013 16:06:37 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Sat, 14 Dec 2013 16:06:37 -0800 (PST) In-Reply-To: References: Date: Sat, 14 Dec 2013 16:06:37 -0800 X-Google-Sender-Auth: tuzNx1qdLgpkxQuangBtsdsaLOE Message-ID: Subject: Re: buf_ring in HEAD is racy From: Adrian Chadd To: Ryan Stone Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Dec 2013 00:06:38 -0000 oh cool, you just did the output-drops thing I was about to code up. We're missing those counters at work and the ops guys poked me about it. I'll also give that a whirl locally and see about working with jack to get it into -HEAD / MFC'ed to 10. Thanks! -a On 13 December 2013 21:04, Ryan Stone wrote: > I am seeing spurious output packet drops that appear to be due to > insufficient memory barriers in buf_ring. I believe that this is the > scenario that I am seeing: > > 1) The buf_ring is empty, br_prod_head = br_cons_head = 0 > 2) Thread 1 attempts to enqueue an mbuf on the buf_ring. It fetches > br_prod_head (0) into a local variable called prod_head > 3) Thread 2 enqueues an mbuf on the buf_ring. The sequence of events > is essentially: > > Thread 2 claims an index in the ring and atomically sets br_prod_head (say to 1) > Thread 2 sets br_ring[1] = mbuf; > Thread 2 does a full memory barrier > Thread 2 updates br_prod_tail to 1 > > 4) Thread 2 dequeues the packet from the buf_ring using the > single-consumer interface. The sequence of events is essentialy: > > Thread 2 checks whether queue is empty (br_cons_head == br_prod_tail), > this is false > Thread 2 sets br_cons_head to 1 > Thread 2 grabs the mbuf from br_ring[1] > Thread 2 sets br_cons_tail to 1 > > 5) Thread 1, which is still attempting to enqueue an mbuf on the ring. > fetches br_cons_tail (1) into a local variable called cons_tail. It > sees cons_tail == 1 but prod_head == 0 and concludes that the ring is > full and drops the packet (incrementing br_drops unatomically, I might > add) > > > I can reproduce several drops per minute by configuring the ixgbe > driver to use only 1 queue and then sending traffic from concurrent 8 > iperf processes. (You will need this hacky patch to even see the > drops with netstat, though: > http://people.freebsd.org/~rstone/patches/ixgbe_br_drops.diff) > > I am investigating fixing buf_ring by using acquire/release semantics > rather than load/store barriers. However, I note that this will > apparently be the second attempt to fix buf_ring, and I'm seriously > questioning whether this is worth the effort compared to the > simplicity of using a mutex. I'm not even convinced that a correct > lockless implementation will even be a performance win, given the > number of memory barriers that will apparently be necessary. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Dec 15 00:08:14 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1AC1D1E4 for ; Sun, 15 Dec 2013 00:08:14 +0000 (UTC) Received: from mail-qc0-x22c.google.com (mail-qc0-x22c.google.com [IPv6:2607:f8b0:400d:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CE5131887 for ; Sun, 15 Dec 2013 00:08:13 +0000 (UTC) Received: by mail-qc0-f172.google.com with SMTP id e16so2734832qcx.31 for ; Sat, 14 Dec 2013 16:08:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=PyfrfCfc/HJ0xE127FH5YhaadIdrjxpmDCD5CglniHQ=; b=RBrl7ONZQeOCxeDJDmbzEO9AVjmt+yFTqjEMYwXwSX4b2PwEs7dl6wh9MMIr9qEL9b NdRa1uxuL+KxvMP70KqLGEuvkCkEhOFHFYrsdHiL+C21ID83gkEaYBpTDVmb0D//EPCv OlJT2byNxKzE1GvsYMZ72NULN+Fhe0awoHky2H3KTnTaYhN4mZ5In5qYQ4ldur4u5YUo 5vVOb6rYLgVYuqdsLJKVlFpbAPtQG1I+sK05WNMIqBX8PcCJqvpGzL7gUzx0xClKXBU8 mxJUVa+gaTOr7Eyu/bO6eg0sSVN7lffICweCSim1tXnq5eYEcC1Xmfjlm6t6vgh9p5sF x+oQ== MIME-Version: 1.0 X-Received: by 10.49.129.38 with SMTP id nt6mr19104063qeb.78.1387066092954; Sat, 14 Dec 2013 16:08:12 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Sat, 14 Dec 2013 16:08:12 -0800 (PST) In-Reply-To: References: Date: Sat, 14 Dec 2013 16:08:12 -0800 X-Google-Sender-Auth: 0im3w8BUl5JfKvn9H0DhSfuXQOQ Message-ID: Subject: Re: buf_ring in HEAD is racy From: Adrian Chadd To: Ryan Stone Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Dec 2013 00:08:14 -0000 Also, I think it's a good idea to get a bullet-proof bufring implementation that uses atomics rather than mutexes. We really want there to be a cheap way to insert things into the queue. Honestly, we could just do with a generic multi-writer, single-reader ring type that we can re-use for other things, rather than just mbufs. We could also have the code run in userland and do some very in-depth stress testing of it there rather than needing it to be in the kernel. Hm.. -adrian From owner-freebsd-net@FreeBSD.ORG Sun Dec 15 04:46:36 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 63523510 for ; Sun, 15 Dec 2013 04:46:36 +0000 (UTC) Received: from mail-qc0-x230.google.com (mail-qc0-x230.google.com [IPv6:2607:f8b0:400d:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0C47A17A9 for ; Sun, 15 Dec 2013 04:46:35 +0000 (UTC) Received: by mail-qc0-f176.google.com with SMTP id i8so2768324qcq.7 for ; Sat, 14 Dec 2013 20:46:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=8ZXzW1ZCm9/N+f6mA6xMy8/SxdKJv5OdQyWv60wFsXE=; b=sXhowColNR3WCVt1quLlkKvnUcw7tx6kvAi9Grz8sOU1jqAM/4WlmUzq4VBcBOlanb 4vE24sPQ3AQYvMX+efhSXYomkh9VpuZ3Oln1+4GN9rM4hgVrT/hHc0yi+Iurv6iMfnZ0 5kjIOx2XioLm8mwpT87yRg8TUkIWvU+w+L+5SG19fMRC1vFpHZywEFIPLdbpGXAuCvQp mRP6NSejZBiO84Yv+wgF+ilwimKorueTVy1kkg8wYX/IY10Dzeq0/d7ZGRE5CUzWF9I3 N1UmZj4a8taK6LWSouEI5JjoSjIWJdFP8pSR9CVAhF38vEtB5/iJ0N0PtzNNONGlet2W aVeQ== MIME-Version: 1.0 X-Received: by 10.224.136.136 with SMTP id r8mr20659560qat.0.1387082795198; Sat, 14 Dec 2013 20:46:35 -0800 (PST) Sender: shteryana@gmail.com Received: by 10.224.128.72 with HTTP; Sat, 14 Dec 2013 20:46:35 -0800 (PST) In-Reply-To: References: Date: Sun, 15 Dec 2013 06:46:35 +0200 X-Google-Sender-Auth: lIi-vzlmj8w6YdzhqGphlYwS2YY Message-ID: Subject: Re: RTL8111/8168B NIC & FreeBSD From: Shteryana Shopova To: David Rufino Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@FreeBSD.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: syrinx@FreeBSD.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Dec 2013 04:46:36 -0000 Hi, I hit the same problem with the same chip on FreeBSD 11.0-CURRENT #0 r258178 - the card would receive nothing or (perhaps not-so-)random crap. ifconfig down/up and disabling RX/TX check-summing makes the card work for me as a temporary workaround. re0@pci0:4:0:1: class=0x020000 card=0x07981025 chip=0x816810ec rev=0x14 hdr=0x00 vendor = 'Realtek Semiconductor Co., Ltd.' device = 'RTL8111/8168B PCI Express Gigabit Ethernet controller' class = network subclass = ethernet cheers, Shteryana On Sat, Dec 14, 2013 at 11:31 PM, David Rufino wrote: > Hello, > > I'm having similar problems using an onboard RTL 8168B NIC. I noticed > there were some recent changes to the re0 driver, so I tried -STABLE > (r258961). The device is detected but can't send or receive packets. > Interestingly it doesn't work in Debian wheezy, but does work fine in > Windows 7 with latest drivers. I'm happy to assist in any way. Here's > some relevant debugging information > > pciconf -l -v > > re0@pci0:1:0:0: class=0x020000 card=0x81161019 chip=0x816810ec rev=0x0c hdr=0x00 > vendor = 'Realtek Semiconductor Co., Ltd.' > device = 'RTL8111/8168B PCI Express Gigabit Ethernet controller' > class = network > subclass = ethernet > > re0: port > 0xe000-0xe0ff mem 0xfea00000-0xfea00fff,0xd0800000-0xd0803fff irq 32 > at device 0.0 on pci1 > re0: Using 1 MSI-X message > re0: ASPM disabled > re0: Chip rev. 0x4c000000 > re0: MAC rev. 0x00000000 > miibus0: on re0 > re0: Ethernet address: 74:27:ea:d3:de:5a > > Thanks!, > David > >> >> Hello, >> >> we are trying to install FreeBSD on a computer that uses the NIC mentioned >> above. The NIC is running under linux without problems, which we've tested >> for several days transferring several GB of data. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Dec 15 06:40:24 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 15FB43EE; Sun, 15 Dec 2013 06:40:24 +0000 (UTC) Received: from mail-ob0-x22d.google.com (mail-ob0-x22d.google.com [IPv6:2607:f8b0:4003:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C6CD21DDA; Sun, 15 Dec 2013 06:40:23 +0000 (UTC) Received: by mail-ob0-f173.google.com with SMTP id gq1so3614046obb.32 for ; Sat, 14 Dec 2013 22:40:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=3gdBQEsRrmSiEF4ls81Jpc1bNySSkAZB7VHPLS/dvUQ=; b=Gq0AeF9HANoE7NWwzziQ0LyuUiM9qSp+qroT23T+R+noNWLA6Htc0kOWKsRRrTV2or fOyv3kqWJqNcjctixIi2rXFrKc0nEhjqIaPtCrF71gznYL81N+X9O7FSdjFWHoE2qcxe iKDRyxscCIgaHMKa5DhzcV92mqSYW+T68lHsebxGFnerUgCs5lbaixXoL2MzA/AWtqI4 KJ7TBHgg0dZuNf7HuvRMSlr3t9L4bLAYn8G/XUIROEpmhKj6OmoNC4byidup8EqgEbE2 39E70RR4Tijc2vKiuXYsfgy8QJl3ZvtBf0AaHM+Lj8g/V4Y3lfjIw52WmYtYPIlCPYPP DkDw== MIME-Version: 1.0 X-Received: by 10.182.24.69 with SMTP id s5mr7432255obf.35.1387089622480; Sat, 14 Dec 2013 22:40:22 -0800 (PST) Received: by 10.76.158.225 with HTTP; Sat, 14 Dec 2013 22:40:22 -0800 (PST) In-Reply-To: References: Date: Sun, 15 Dec 2013 01:40:22 -0500 Message-ID: Subject: Re: buf_ring in HEAD is racy From: Ryan Stone To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Dec 2013 06:40:24 -0000 On Sat, Dec 14, 2013 at 7:06 PM, Adrian Chadd wrote: > oh cool, you just did the output-drops thing I was about to code up. > We're missing those counters at work and the ops guys poked me about > it. > > I'll also give that a whirl locally and see about working with jack to > get it into -HEAD / MFC'ed to 10. > > Thanks! The reason why I call it hacky is because these drops are properly categorized as discards, not errors. But there's not currently a really good way to distinguish between the two in FreeBSD right now (there is the drops counter in the if_snd queue but netstat seemed to be ignoring that even with -d specified and blindly overwriting it would seem to be incompatible with altq anyway) From owner-freebsd-net@FreeBSD.ORG Sun Dec 15 06:41:40 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 39D83480; Sun, 15 Dec 2013 06:41:40 +0000 (UTC) Received: from mail-ob0-x22d.google.com (mail-ob0-x22d.google.com [IPv6:2607:f8b0:4003:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EA4701E27; Sun, 15 Dec 2013 06:41:39 +0000 (UTC) Received: by mail-ob0-f173.google.com with SMTP id gq1so3595268obb.4 for ; Sat, 14 Dec 2013 22:41:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=2bK2hJGTglF2kVuY03nUv/Z7+anDWk4U53ZlzL5VMeY=; b=QBLoUrCjtsikj2GsiiYYQQDe3arIKgnoWkajAHQMFvrwGKuvABew42W2wnLiTEDK+S wkI3yRZlfXJAMRcK2XvTPwqQclhJy5BtrBOYS9Mlyp+b6++pIt+3mr1pezYHLUOIdtrI 9RlL0PFW+LM6Nq0kGVop/ePOhXVOxu6faN+YXuYb9P6TY7MhOuWIoKObz2SLM8kKeSxd 9W4xbyh94S50bc7/OLh+pPMhmtioK8FwK9WJruWYETFDPCUee9syN2is1XW6Up/1WAvi 7h1armg+c1H9T1Jtdg0Ggo03XsKwc9h/rKLtCOfDjIGn8nVgJbacwoab/d2xWYFTVdHo aN4g== MIME-Version: 1.0 X-Received: by 10.182.241.8 with SMTP id we8mr1967382obc.62.1387089699319; Sat, 14 Dec 2013 22:41:39 -0800 (PST) Received: by 10.76.158.225 with HTTP; Sat, 14 Dec 2013 22:41:39 -0800 (PST) In-Reply-To: References: Date: Sun, 15 Dec 2013 01:41:39 -0500 Message-ID: Subject: Re: buf_ring in HEAD is racy From: Ryan Stone To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Dec 2013 06:41:40 -0000 On Sat, Dec 14, 2013 at 7:08 PM, Adrian Chadd wrote: > Honestly, we could just do with a generic multi-writer, single-reader > ring type that we can re-use for other things, rather than just mbufs. That is actually exactly what buf_ring is -- it deals with void *, not struct mbuf *. > We could also have the code run in userland and do some very in-depth > stress testing of it there rather than needing it to be in the kernel. For testing purposes this isn't very hard -- I did it yesterday while debugging this problem. The one trick is that buf_ring uses critical_enter/critical_exit to prevent livelock (which was woefully undocumented in the code) so when I was testing I had to make sure that I didn't have more threads than cores. From owner-freebsd-net@FreeBSD.ORG Sun Dec 15 20:35:41 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F8C56AF for ; Sun, 15 Dec 2013 20:35:41 +0000 (UTC) Received: from mail-qa0-x229.google.com (mail-qa0-x229.google.com [IPv6:2607:f8b0:400d:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CEF36102F for ; Sun, 15 Dec 2013 20:35:40 +0000 (UTC) Received: by mail-qa0-f41.google.com with SMTP id j5so988287qaq.7 for ; Sun, 15 Dec 2013 12:35:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=Wokod05gb6bntYMV0x2p45Oc8Yh3TpOtK/s/GuNU+d0=; b=shjibnu3r9Ng21klJd1R6NqKDew4YNNdSfAt+V5jkvzEn1qHrBH8OSWTI5eFLfM3fW rDE+v64cenlk3CS6hdLqWGNnXaUbNtRHvIXrED3oLDPc/KDT5tAo5qKzH1/UL3IlNHeR wR52mr9UQxP9m2Mg+MtyDbWr/spHm6qudDVMWHusnlpu3rx0gOqEGuJrCkjDAosfvkbW oIQ9qoOscTGVMa9XqMjCXxyolO6JD41aROOYiRnscztPayXS1oMRpdj7Hlddr8rCs97d d8VhYpP2oPmux+JLm7/IkS3TFSDnNNa1ewbmt3SAIzEjv5DCFmILA1wm9wEwDAo6/pDM Rxog== MIME-Version: 1.0 X-Received: by 10.224.67.200 with SMTP id s8mr26107582qai.75.1387139739878; Sun, 15 Dec 2013 12:35:39 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Sun, 15 Dec 2013 12:35:39 -0800 (PST) In-Reply-To: References: <523457A1.3090606@debian.org> Date: Sun, 15 Dec 2013 12:35:39 -0800 X-Google-Sender-Auth: 5boT7rSYKhzLlUfjlKTTsR8nrpA Message-ID: Subject: Re: IPSEC From: Adrian Chadd To: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Eitan Adler , Robert Millan , "debian-bsd@lists.debian.org" , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Dec 2013 20:35:41 -0000 If it's actually improving performance then it's likely some kind of data / code alignment / caching interplay. It'd be a good project to hunt that down and figure out what's going on. -a On 8 December 2013 11:02, Olivier Cochard-Labb=E9 wrot= e: > On Sun, Dec 8, 2013 at 12:16 AM, Eitan Adler wrote= : >> Hi all, >> >> I understand this is an old thread but I do not see an answer here. >> Can anyone answer the question below? >> >> On Sat, Sep 14, 2013 at 8:33 AM, Robert Millan wrote: >>> >>> Hi! >>> >>> Is there any particular reason (performance, stability concerns...) >>> IPSEC support is not enabled in GENERIC? >>> >>> In Debian GNU/kFreeBSD we're considering enabling it in our default >>> builds, due to increased user demand and as it is already enabled for >>> our Linux-based flavours. >>> >>> However we're concerned about diverging from FreeBSD as there might be >>> unforeseen consequences. Is there any specific concern on your side? >>> >>> If not, perhaps it could be considered for HEAD after 10.0 release? >> >> > > Here are my own bench result regarding forwarding speed (paquet-per-secon= d) > with a kernel compiled without-ipsec and with ipsec (ipsec is not enabled > during the tests, just present on the kernel) of FreeBSD 10.0-PRERELEASE: > > ministat -s without-ipsec ipsec > x without-ipsec > + ipsec > +------------------------------------------------------------------------= --------+ > |x + x + +x x x + > +| > | |__________________A_____M____________| > | > | |_______________M_________A__________________________| > | > +------------------------------------------------------------------------= --------+ > N Min Max Median Avg Stdd= ev > x 5 1646075 1764528 1725461 1713080 44560.0= 59 > + 5 1685034 1833206 1724461 1748666.8 62356.2= 18 > No difference proven at 95.0% confidence > > I didn't see negative impact of enabling ipsec (it's even a little bit > better with it). > > Regards, > > Olivier > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sun Dec 15 20:46:04 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5672790A for ; Sun, 15 Dec 2013 20:46:04 +0000 (UTC) Received: from mail-qc0-x233.google.com (mail-qc0-x233.google.com [IPv6:2607:f8b0:400d:c01::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 149E010BB for ; Sun, 15 Dec 2013 20:46:04 +0000 (UTC) Received: by mail-qc0-f179.google.com with SMTP id i8so3077921qcq.24 for ; Sun, 15 Dec 2013 12:46:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=JuxmNBUu0VsvDD3NVTV5sV7Cg1jICcjlaHRwezRCtCI=; b=0ohmZeYzgjol/Z3Bsba+/zQaa5c7n71JDVufKdHQu3/MqLryQOylsuSnDHtFW57tO/ 0EHehs2HVNT8P4k+kREomJAvFHWt+VNYkoLx87AG/XdumQZOBBOhPYf/LKMZuS601AZr IwqWGlQk7Jw4Nbo7CcQqFSO3nDYAzuAPR/ijh4OSgWhL5NXbw+gBhIgQETbNqUwjUZ9/ SWjLtx40KAsWcNrdeamDQFQnfhXbcTcjvW44NENDMjwFtL556UMmzyEAWWqPRKJ6thMF uC9eai6IWb19iv0J03kufjMR6uF5MiCWtl4HM9URZpKGxN4eVjVTBous3DAPGWEzQPff YKZg== MIME-Version: 1.0 X-Received: by 10.229.53.68 with SMTP id l4mr26088838qcg.21.1387140363257; Sun, 15 Dec 2013 12:46:03 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Sun, 15 Dec 2013 12:46:03 -0800 (PST) In-Reply-To: References: Date: Sun, 15 Dec 2013 12:46:03 -0800 X-Google-Sender-Auth: 0_XD_XVBtSV5L4f5PInF2sQQANA Message-ID: Subject: Re: buf_ring in HEAD is racy From: Adrian Chadd To: Ryan Stone Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Dec 2013 20:46:04 -0000 On 14 December 2013 22:40, Ryan Stone wrote: > On Sat, Dec 14, 2013 at 7:06 PM, Adrian Chadd wrote: >> oh cool, you just did the output-drops thing I was about to code up. >> We're missing those counters at work and the ops guys poked me about >> it. >> >> I'll also give that a whirl locally and see about working with jack to >> get it into -HEAD / MFC'ed to 10. >> >> Thanks! > > The reason why I call it hacky is because these drops are properly > categorized as discards, not errors. But there's not currently a > really good way to distinguish between the two in FreeBSD right now > (there is the drops counter in the if_snd queue but netstat seemed to > be ignoring that even with -d specified and blindly overwriting it > would seem to be incompatible with altq anyway) Hm. A few drivers (eg cxgbe) update the ifsnd drops counter. oh, holy crap! How'd I never notice that the drops column is empty? How about I just go ahead and add that field to the ifnet statistics and then teach netstat how to display them? What do you think? Then we can "fix" the other drivers such as cxgbe (for reasons surrounding "we use this at work" :) so they can properly use this? -adrian From owner-freebsd-net@FreeBSD.ORG Sun Dec 15 20:47:09 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A27F89AA for ; Sun, 15 Dec 2013 20:47:09 +0000 (UTC) Received: from mail-qe0-x22c.google.com (mail-qe0-x22c.google.com [IPv6:2607:f8b0:400d:c02::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5FFCD10C9 for ; Sun, 15 Dec 2013 20:47:09 +0000 (UTC) Received: by mail-qe0-f44.google.com with SMTP id nd7so3322515qeb.31 for ; Sun, 15 Dec 2013 12:47:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=8wrtBR0u1GU6pgLkShBAUtVNnUmWPYxnByQVdGAd7qU=; b=wqQ0mcdTQWHsTcKiy7mQslUZkWTpwaQPSlcpaI+TweXpupBS1q4CYHsEkJOc4hb/tN HPjh7Ti5Q06qyeqcnfqnVnUMSjKLnx/SWS/+Et5Z5aPPxyd+83kdvylqqjCT2UUDMbVe /qhTUubv+Pq2z1NOZMC1vtiVFaxWv68GTggXyyMytkfi83JaR9JXx3VEemVZOW0ixuKn TSukx+CQav9F6esuCmuKabct/UtnVN4UcAT4j17rXG0xqd+VTWdeAAo0jbBV9qDJil9i XL3DsvnNEnuw5IWctdpA1gAhT3BZ2/XG4878AT4iLsJ2Pit+9OzXZKq2+AqewtqZU4Xj OSNQ== MIME-Version: 1.0 X-Received: by 10.49.17.232 with SMTP id r8mr26487666qed.74.1387140428618; Sun, 15 Dec 2013 12:47:08 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Sun, 15 Dec 2013 12:47:08 -0800 (PST) In-Reply-To: References: Date: Sun, 15 Dec 2013 12:47:08 -0800 X-Google-Sender-Auth: U_sH3bjOT5bKzu3qpz6lbG6sr_A Message-ID: Subject: Re: buf_ring in HEAD is racy From: Adrian Chadd To: Ryan Stone Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Dec 2013 20:47:09 -0000 surely we can do this without critical enter/exit.. How's this implementation of single-reader multiple-writer compare to like, the dozens of other userland ones that are out there in the open source world? -a On 14 December 2013 22:41, Ryan Stone wrote: > On Sat, Dec 14, 2013 at 7:08 PM, Adrian Chadd wrote: >> Honestly, we could just do with a generic multi-writer, single-reader >> ring type that we can re-use for other things, rather than just mbufs. > > That is actually exactly what buf_ring is -- it deals with void *, not > struct mbuf *. > >> We could also have the code run in userland and do some very in-depth >> stress testing of it there rather than needing it to be in the kernel. > > For testing purposes this isn't very hard -- I did it yesterday while > debugging this problem. The one trick is that buf_ring uses > critical_enter/critical_exit to prevent livelock (which was woefully > undocumented in the code) so when I was testing I had to make sure > that I didn't have more threads than cores. From owner-freebsd-net@FreeBSD.ORG Sun Dec 15 21:23:28 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 897DCE82 for ; Sun, 15 Dec 2013 21:23:28 +0000 (UTC) Received: from mail-pb0-x22d.google.com (mail-pb0-x22d.google.com [IPv6:2607:f8b0:400e:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5736B12BC for ; Sun, 15 Dec 2013 21:23:28 +0000 (UTC) Received: by mail-pb0-f45.google.com with SMTP id rp16so4622176pbb.18 for ; Sun, 15 Dec 2013 13:23:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=U2OK4SfX5oqzp9M1Be4CCiT2FXzCQ8JXC51ZI4L58d8=; b=OBKzN5Dui4hIpmvvNtiwFS7oHHo/s9atTawhSXSq7fCniaZpTd1insZ/NaKd1m2Ebe 56LRYLoHe+MF/NIAjrdglTkXuF+bfpi7GzmfTpKRPOMbS6xaogLrmy6Ywx098k64+Vim wwyZoLSZ2jPMRQ2Kvh0klngK5Q0XX0pov+XY4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=U2OK4SfX5oqzp9M1Be4CCiT2FXzCQ8JXC51ZI4L58d8=; b=SUtCxbLogxC46ggY4zi5KU2cOx20g6nTjzPfm8ioOpz4tN4tO+9lS3kmx7qA7vYqVB RYUD1sHnotjDQU7A6jxqml7OHanNDgjhq/YqdbhffStnTcqsGNvtvmsSjhkGtaq5EgvV ROkz5dFQbilsiuvKfH8N1/Goly5lC4Asf3jksrT9+BE6a2QN1rjIrRWvZJUUgMm5C0HU wHucdMg6SPabWPzOd/zOva9i3XSUKqs+QKs9BNBtJAYGPb4xfRNIEmuR0LqmpUwttDFE MSzBClwxUQx0u5WbavISXbnwxnJWde5sRfS7ZjrsLBWLnjz2bIOelSOl+xEo6bDyQiUs r8UQ== X-Gm-Message-State: ALoCoQkXJbwBAMkX3Fy3CO/GJ9xmzFUwsuLuAMqsPlz9X/lLLEidghjlhuebwbxmY3o0+aFIxMMb X-Received: by 10.66.221.103 with SMTP id qd7mr16213744pac.44.1387142607894; Sun, 15 Dec 2013 13:23:27 -0800 (PST) MIME-Version: 1.0 Received: by 10.70.57.98 with HTTP; Sun, 15 Dec 2013 13:22:57 -0800 (PST) In-Reply-To: References: <523457A1.3090606@debian.org> From: Eitan Adler Date: Sun, 15 Dec 2013 16:22:57 -0500 Message-ID: Subject: Re: IPSEC To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 Cc: =?UTF-8?Q?Olivier_Cochard=2DLabb=C3=A9?= , "freebsd-net@freebsd.org" , Robert Millan , "debian-bsd@lists.debian.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Dec 2013 21:23:28 -0000 On Sun, Dec 15, 2013 at 3:35 PM, Adrian Chadd wrote: > If it's actually improving performance then it's likely some kind of > data / code alignment / caching interplay. It'd be a good project to > hunt that down and figure out what's going on. >From the data presented it *not* improving performance. I received private mail explaining that it was racy and would panic a server under heavy load. From owner-freebsd-net@FreeBSD.ORG Sun Dec 15 21:31:04 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 69651E4 for ; Sun, 15 Dec 2013 21:31:04 +0000 (UTC) Received: from olgeni.olgeni.com (host-156-246-171-31.cloudsigma.com [31.171.246.156]) by mx1.freebsd.org (Postfix) with ESMTP id 2DAED133C for ; Sun, 15 Dec 2013 21:31:03 +0000 (UTC) Received: from olgeni.olgeni (unknown [82.84.68.101]) by olgeni.olgeni.com (Postfix) with ESMTPSA id C0A17174491; Sun, 15 Dec 2013 22:24:16 +0100 (CET) Date: Sun, 15 Dec 2013 22:24:16 +0100 (CET) From: Jimmy Olgeni X-X-Sender: olgeni@olgeni.olgeni To: Eugene Perevyazko Subject: Re: ipsec packets apparently not getting to destination In-Reply-To: <20131204122115.GA46835@traktor.dnepro.net> Message-ID: References: <20131204122115.GA46835@traktor.dnepro.net> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Dec 2013 21:31:04 -0000 Hi, On Wed, 4 Dec 2013, Eugene Perevyazko wrote: > mpd uses netgraph for most if not all processing. Could it be that > ipsec-processed packets do not enter corresponding netgraph node? You can > look at the netgraph tree to see where mpd expects to see incoming > packets. Opening an UDP listening port with socat(1) does not show any traffic, which (unfortunately) excludes weird paths inside netgraph. It just doesn't get through... -- jimmy From owner-freebsd-net@FreeBSD.ORG Mon Dec 16 01:41:07 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 15854ABE for ; Mon, 16 Dec 2013 01:41:07 +0000 (UTC) Received: from mail-pd0-x22e.google.com (mail-pd0-x22e.google.com [IPv6:2607:f8b0:400e:c02::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DCF8D1472 for ; Mon, 16 Dec 2013 01:41:06 +0000 (UTC) Received: by mail-pd0-f174.google.com with SMTP id y13so4653181pdi.19 for ; Sun, 15 Dec 2013 17:41:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=mtYOmqTLPAh8YrNBN/MlxgSHVFAZOGdR5KMNXkEjdmk=; b=Z8DFEpIQGkYpZ05L+R3VXsIBLGZmLs2NA/Sx4kvlgtJ5RNBd8SxtUEVI+Y7MnYkDen nInGjsQ9iASfXV8IsMzy1FbaQIVNQdHh9fC2Xlj0txTFAYA2yKR1SWctSvpno8t6z3Ps gCx1qfysK8ggzZEKFDxSATQeKFuVd8kj1Yq0ShojMVg6jL5D3lsaO7SnGXu5M5EbhP+C HVuFX5/k8tH5lyOlKXX/Jqkn8KB1ED/o6QTWzN28ERVXCeRmUYhkFJgGC6SXZVBpTxsk 7CnoSkkqjPydmqRnyXjmwxC3L6UEvxxkS3ZrzkLjh4iSI/h/xJdlAo/zdLLGncD/VT1R avIQ== X-Received: by 10.68.217.129 with SMTP id oy1mr17244770pbc.23.1387158066451; Sun, 15 Dec 2013 17:41:06 -0800 (PST) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id g6sm30304975pat.2.2013.12.15.17.41.02 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Sun, 15 Dec 2013 17:41:04 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 16 Dec 2013 10:40:59 +0900 From: Yonghyeon PYUN Date: Mon, 16 Dec 2013 10:40:58 +0900 To: David Rufino Subject: Re: RTL8111/8168B NIC & FreeBSD Message-ID: <20131216014058.GA3345@michelle.cdnetworks.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 01:41:07 -0000 On Sat, Dec 14, 2013 at 04:31:56PM -0500, David Rufino wrote: > Hello, > > I'm having similar problems using an onboard RTL 8168B NIC. I noticed > there were some recent changes to the re0 driver, so I tried -STABLE > (r258961). The device is detected but can't send or receive packets. > Interestingly it doesn't work in Debian wheezy, but does work fine in > Windows 7 with latest drivers. I'm happy to assist in any way. Here's > some relevant debugging information > > pciconf -l -v > > re0@pci0:1:0:0: class=0x020000 card=0x81161019 chip=0x816810ec rev=0x0c hdr=0x00 > vendor = 'Realtek Semiconductor Co., Ltd.' > device = 'RTL8111/8168B PCI Express Gigabit Ethernet controller' > class = network > subclass = ethernet > > re0: port > 0xe000-0xe0ff mem 0xfea00000-0xfea00fff,0xd0800000-0xd0803fff irq 32 > at device 0.0 on pci1 > re0: Using 1 MSI-X message > re0: ASPM disabled > re0: Chip rev. 0x4c000000 > re0: MAC rev. 0x00000000 > miibus0: on re0 > re0: Ethernet address: 74:27:ea:d3:de:5a > Your controller is RTL8168G, latest generation of RTL8168 family. I don't see issues you mentioned above on RTL8168 engineering sample board. Different motherboard vendors shall use different H/W configuration so that may trigger that issue. I don't have any clue about that but I'll let you know if I manage to narrow down the issue. > Thanks!, > David > > > > > Hello, > > > > we are trying to install FreeBSD on a computer that uses the NIC mentioned > > above. The NIC is running under linux without problems, which we've tested > > for several days transferring several GB of data. From owner-freebsd-net@FreeBSD.ORG Mon Dec 16 01:47:20 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 344D5B8A; Mon, 16 Dec 2013 01:47:20 +0000 (UTC) Received: from mail-pa0-x232.google.com (mail-pa0-x232.google.com [IPv6:2607:f8b0:400e:c03::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0048714A4; Mon, 16 Dec 2013 01:47:19 +0000 (UTC) Received: by mail-pa0-f50.google.com with SMTP id kl14so2262238pab.23 for ; Sun, 15 Dec 2013 17:47:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=qMTbrLAR4nwZgbDvT6nCxSy8F95x2PLnSjlIktdkpiE=; b=riz+rXzSOQ5o+rSZUOD0MNtLDJtemp4RJWCEHt0Ph1gXQTo/OVdvbD/XVcnL03aKQz w3swPVTEj0ny/QOapiy7Du8BoC5NShRtnzewio7Nc8KgVoyHCMXRVD7Y3/qhzJZv+ULr ZTP9xsjulEemiNewQ1Qwk2ZFfnQbWYok/87mWfUhcFV9YLffubgbNC/7NqONGjJQBMeb dyvoNzQXBhsLraO5EqjdX5L2hi1uE8bMYrzo3fwow9jfu1jIDMDllqpenl9nD8tKDm9i 1WwxsBKDbKBF307+Kl9cC/lucNZFaB51ZhvpHh4ZwS6PdCBWKL4C21xYJMi2A3/y+mh4 W0yg== X-Received: by 10.68.176.65 with SMTP id cg1mr17337101pbc.145.1387158439672; Sun, 15 Dec 2013 17:47:19 -0800 (PST) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id nw11sm30340738pab.13.2013.12.15.17.47.14 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Sun, 15 Dec 2013 17:47:17 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 16 Dec 2013 10:47:04 +0900 From: Yonghyeon PYUN Date: Mon, 16 Dec 2013 10:47:04 +0900 To: Shteryana Shopova Subject: Re: RTL8111/8168B NIC & FreeBSD Message-ID: <20131216014704.GB3345@michelle.cdnetworks.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: "freebsd-net@FreeBSD.org" , David Rufino X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 01:47:20 -0000 On Sun, Dec 15, 2013 at 06:46:35AM +0200, Shteryana Shopova wrote: > Hi, > > I hit the same problem with the same chip on FreeBSD 11.0-CURRENT #0 > r258178 - the card would receive nothing or (perhaps not-so-)random > crap. ifconfig down/up and disabling RX/TX check-summing makes the > card work for me as a temporary workaround. > David's controller is RTL8168G. Are you sure you have the same controller? If your controller is RTL8168E-VL(Chip rev. 0x2c800000 in dmesg), try the patch at the following URL. http://people.freebsd.org/~yongari/re/re.8168evl.diff > re0@pci0:4:0:1: class=0x020000 card=0x07981025 chip=0x816810ec > rev=0x14 hdr=0x00 > vendor = 'Realtek Semiconductor Co., Ltd.' > device = 'RTL8111/8168B PCI Express Gigabit Ethernet controller' > class = network > subclass = ethernet > > cheers, > Shteryana > > > On Sat, Dec 14, 2013 at 11:31 PM, David Rufino wrote: > > Hello, > > > > I'm having similar problems using an onboard RTL 8168B NIC. I noticed > > there were some recent changes to the re0 driver, so I tried -STABLE > > (r258961). The device is detected but can't send or receive packets. > > Interestingly it doesn't work in Debian wheezy, but does work fine in > > Windows 7 with latest drivers. I'm happy to assist in any way. Here's > > some relevant debugging information > > > > pciconf -l -v > > > > re0@pci0:1:0:0: class=0x020000 card=0x81161019 chip=0x816810ec rev=0x0c hdr=0x00 > > vendor = 'Realtek Semiconductor Co., Ltd.' > > device = 'RTL8111/8168B PCI Express Gigabit Ethernet controller' > > class = network > > subclass = ethernet > > > > re0: port > > 0xe000-0xe0ff mem 0xfea00000-0xfea00fff,0xd0800000-0xd0803fff irq 32 > > at device 0.0 on pci1 > > re0: Using 1 MSI-X message > > re0: ASPM disabled > > re0: Chip rev. 0x4c000000 > > re0: MAC rev. 0x00000000 > > miibus0: on re0 > > re0: Ethernet address: 74:27:ea:d3:de:5a > > > > Thanks!, > > David > > > >> > >> Hello, > >> > >> we are trying to install FreeBSD on a computer that uses the NIC mentioned > >> above. The NIC is running under linux without problems, which we've tested > >> for several days transferring several GB of data. From owner-freebsd-net@FreeBSD.ORG Mon Dec 16 11:06:51 2013 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F184DEE8 for ; Mon, 16 Dec 2013 11:06:51 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DBF6C134B for ; Mon, 16 Dec 2013 11:06:51 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id rBGB6pFq019432 for ; Mon, 16 Dec 2013 11:06:51 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id rBGB6pZF019430 for freebsd-net@FreeBSD.org; Mon, 16 Dec 2013 11:06:51 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 16 Dec 2013 11:06:51 GMT Message-Id: <201312161106.rBGB6pZF019430@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-net@FreeBSD.org Subject: Current problem reports assigned to freebsd-net@FreeBSD.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 11:06:52 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/184311 net [bge] [panic] kernel panic with bge(4) on SunFire X210 o kern/184084 net [ral] kernel crash by ral (RT3090) o bin/183687 net [patch] route(8): route add -net 172.20 add wrong host o kern/183659 net [tcp] ]TCP stack lock contention with short-lived conn o conf/183407 net [rc.d] [patch] Routing restart returns non-zero exitco o kern/183391 net [oce] 10gigabit networking problems with Emulex OCE 11 o kern/183390 net [ixgbe] 10gigabit networking problems o kern/182917 net [igb] strange out traffic with igb interfaces o kern/182847 net [netinet6] [patch] Remove dead code o kern/182665 net [wlan] Kernel panic when creating second wlandev. o kern/182382 net [tcp] sysctl to set TCP CC method on BIG ENDIAN system o kern/182297 net [cm] ArcNet driver fails to detect the link address - o kern/182212 net [patch] [ng_mppc] ng_mppc(4) blocks on network errors o kern/181970 net [re] LAN RealtekŪ 8111G is not supported by re driver o kern/181931 net [vlan] [lagg] vlan over lagg over mlxen crashes the ke o kern/181823 net [ip6] [patch] make ipv6 mroute return same errror code o kern/181741 net [kernel] [patch] Packet loss when 'control' messages a o kern/181703 net [re] [patch] Fix Realtek 8111G Ethernet controller not o kern/181657 net [bpf] [patch] BPF_COP/BPF_COPX instruction reservation o kern/181257 net [bge] bge link status change o kern/181236 net [igb] igb driver unstable work o kern/181135 net [netmap] [patch] sys/dev/netmap patch for Linux compat o kern/181131 net [netmap] [patch] sys/dev/netmap memory allocation impr o kern/181006 net [run] [patch] mbuf leak in run(4) driver o kern/180893 net [if_ethersubr] [patch] Packets received with own LLADD o kern/180844 net [panic] [re] Intermittent panic (re driver?) o kern/180775 net [bxe] if_bxe driver broken with Broadcom BCM57711 card o kern/180722 net [bluetooth] bluetooth takes 30-50 attempts to pair to s kern/180468 net [request] LOCAL_PEERCRED support for PF_INET o kern/180065 net [netinet6] [patch] Multicast loopback to own host brok o kern/179926 net [lacp] [patch] active aggregator selection bug o kern/179824 net [ixgbe] System (9.1-p4) hangs on heavy ixgbe network t o kern/179733 net [lagg] [patch] interface loses capabilities when proto o kern/179429 net [tap] STP enabled tap bridge o kern/179299 net [igb] Intel X540-T2 - unstable driver a kern/179264 net [vimage] [pf] Core dump with Packet filter and VIMAGE o kern/178947 net [arp] arp rejecting not working o kern/178782 net [ixgbe] 82599EB SFP does not work with passthrough und o kern/178612 net [run] kernel panic due the problems with run driver o kern/178472 net [ip6] [patch] make return code consistent with IPv4 co o kern/178079 net [tcp] Switching TCP CC algorithm panics on sparc64 wit s kern/178071 net FreeBSD unable to recongize Kontron (Industrial Comput o kern/177905 net [xl] [panic] ifmedia_set when pluging CardBus LAN card o kern/177618 net [bridge] Problem with bridge firewall with trunk ports o kern/177402 net [igb] [pf] problem with ethernet driver igb + pf / alt o kern/177400 net [jme] JMC25x 1000baseT establishment issues o kern/177366 net [ieee80211] negative malloc(9) statistics for 80211nod f kern/177362 net [netinet] [patch] Wrong control used to return TOS o kern/177194 net [netgraph] Unnamed netgraph nodes for vlan interfaces o kern/177184 net [bge] [patch] enable wake on lan o kern/177139 net [igb] igb drops ethernet ports 2 and 3 o kern/176884 net [re] re0 flapping up/down o kern/176671 net [epair] MAC address for epair device not unique o kern/176484 net [ipsec] [enc] [patch] panic: IPsec + enc(4); device na o kern/176446 net [netinet] [patch] Concurrency in ixgbe driving out-of- o kern/176420 net [kernel] [patch] incorrect errno for LOCAL_PEERCRED o kern/176419 net [kernel] [patch] socketpair support for LOCAL_PEERCRED o kern/176401 net [netgraph] page fault in netgraph o kern/176167 net [ipsec][lagg] using lagg and ipsec causes immediate pa o kern/176027 net [em] [patch] flow control systcl consistency for em dr o kern/176026 net [tcp] [patch] TCP wrappers caused quite a lot of warni o kern/175864 net [re] Intel MB D510MO, onboard ethernet not working aft o kern/175852 net [amd64] [patch] in_cksum_hdr() behaves differently on o kern/175734 net no ethernet detected on system with EG20T PCH chipset o kern/175267 net [pf] [tap] pf + tap keep state problem o kern/175236 net [epair] [gif] epair and gif Devices On Bridge o kern/175182 net [panic] kernel panic on RADIX_MPATH when deleting rout o kern/175153 net [tcp] will there miss a FIN when do TSO? o kern/174959 net [net] [patch] rnh_walktree_from visits spurious nodes o kern/174958 net [net] [patch] rnh_walktree_from makes unreasonable ass o kern/174897 net [route] Interface routes are broken o kern/174851 net [bxe] [patch] UDP checksum offload is wrong in bxe dri o kern/174850 net [bxe] [patch] bxe driver does not receive multicasts o kern/174849 net [bxe] [patch] bxe driver can hang kernel when reset o kern/174822 net [tcp] Page fault in tcp_discardcb under high traffic o kern/174602 net [gif] [ipsec] traceroute issue on gif tunnel with ipse o kern/174535 net [tcp] TCP fast retransmit feature works strange o kern/173871 net [gif] process of 'ifconfig gif0 create hangs' when if_ o kern/173475 net [tun] tun(4) stays opened by PID after process is term o kern/173201 net [ixgbe] [patch] Missing / broken ixgbe sysctl's and tu o kern/173137 net [em] em(4) unable to run at gigabit with 9.1-RC2 o kern/173002 net [patch] data type size problem in if_spppsubr.c o kern/172895 net [ixgb] [ixgbe] do not properly determine link-state o kern/172683 net [ip6] Duplicate IPv6 Link Local Addresses o kern/172675 net [netinet] [patch] sysctl_tcp_hc_list (net.inet.tcp.hos p kern/172113 net [panic] [e1000] [patch] 9.1-RC1/amd64 panices in igb(4 o kern/171840 net [ip6] IPv6 packets transmitting only on queue 0 o kern/171739 net [bce] [panic] bce related kernel panic o kern/171711 net [dummynet] [panic] Kernel panic in dummynet o kern/171532 net [ndis] ndis(4) driver includes 'pccard'-specific code, o kern/171531 net [ndis] undocumented dependency for ndis(4) o kern/171524 net [ipmi] ipmi driver crashes kernel by reboot or shutdow s kern/171508 net [epair] [request] Add the ability to name epair device o kern/171228 net [re] [patch] if_re - eeprom write issues o kern/170701 net [ppp] killl ppp or reboot with active ppp connection c o kern/170267 net [ixgbe] IXGBE_LE32_TO_CPUS is probably an unintentiona o kern/170081 net [fxp] pf/nat/jails not working if checksum offloading o kern/169898 net ifconfig(8) fails to set MTU on multiple interfaces. o kern/169676 net [bge] [hang] system hangs, fully or partially after re o kern/169620 net [ng] [pf] ng_l2tp incoming packet bypass pf firewall o kern/169459 net [ppp] umodem/ppp/3g stopped working after update from o kern/169438 net [ipsec] ipv4-in-ipv6 tunnel mode IPsec does not work p kern/168294 net [ixgbe] [patch] ixgbe driver compiled in kernel has no o kern/168246 net [em] Multiple em(4) not working with qemu o kern/168245 net [arp] [regression] Permanent ARP entry not deleted on o kern/168244 net [arp] [regression] Unable to manually remove permanent o kern/168183 net [bce] bce driver hang system o kern/167603 net [ip] IP fragment reassembly's broken: file transfer ov o kern/167500 net [em] [panic] Kernel panics in em driver o kern/167325 net [netinet] [patch] sosend sometimes return EINVAL with o kern/167202 net [igmp]: Sending multiple IGMP packets crashes kernel o kern/166462 net [gre] gre(4) when using a tunnel source address from c o kern/166285 net [arp] FreeBSD v8.1 REL p8 arp: unknown hardware addres o kern/166255 net [net] [patch] It should be possible to disable "promis p kern/165903 net mbuf leak o kern/165622 net [ndis][panic][patch] Unregistered use of FPU in kernel s kern/165562 net [request] add support for Intel i350 in FreeBSD 7.4 o kern/165526 net [bxe] UDP packets checksum calculation whithin if_bxe o kern/165488 net [ppp] [panic] Fatal trap 12 jails and ppp , kernel wit o kern/165305 net [ip6] [request] Feature parity between IP_TOS and IPV6 o kern/165296 net [vlan] [patch] Fix EVL_APPLY_VLID, update EVL_APPLY_PR o kern/165181 net [igb] igb freezes after about 2 weeks of uptime o kern/165174 net [patch] [tap] allow tap(4) to keep its address on clos o kern/165152 net [ip6] Does not work through the issue of ipv6 addresse o kern/164495 net [igb] connect double head igb to switch cause system t o kern/164490 net [pfil] Incorrect IP checksum on pfil pass from ip_outp o kern/164475 net [gre] gre misses RUNNING flag after a reboot o kern/164265 net [netinet] [patch] tcp_lro_rx computes wrong checksum i o kern/163903 net [igb] "igb0:tx(0)","bpf interface lock" v2.2.5 9-STABL o kern/163481 net freebsd do not add itself to ping route packet o kern/162927 net [tun] Modem-PPP error ppp[1538]: tun0: Phase: Clearing o kern/162558 net [dummynet] [panic] seldom dummynet panics o kern/162153 net [em] intel em driver 7.2.4 don't compile o kern/162110 net [igb] [panic] RELENG_9 panics on boot in IGB driver - o kern/162028 net [ixgbe] [patch] misplaced #endif in ixgbe.c o kern/161277 net [em] [patch] BMC cannot receive IPMI traffic after loa o kern/160873 net [igb] igb(4) from HEAD fails to build on 7-STABLE o kern/160750 net Intel PRO/1000 connection breaks under load until rebo o kern/160693 net [gif] [em] Multicast packet are not passed from GIF0 t o kern/160293 net [ieee80211] ppanic] kernel panic during network setup o kern/160206 net [gif] gifX stops working after a while (IPv6 tunnel) o kern/159817 net [udp] write UDPv4: No buffer space available (code=55) o kern/159629 net [ipsec] [panic] kernel panic with IPsec in transport m o kern/159621 net [tcp] [panic] panic: soabort: so_count o kern/159603 net [netinet] [patch] in_ifscrubprefix() - network route c o kern/159601 net [netinet] [patch] in_scrubprefix() - loopback route re o kern/159294 net [em] em watchdog timeouts o kern/159203 net [wpi] Intel 3945ABG Wireless LAN not support IBSS o kern/158930 net [bpf] BPF element leak in ifp->bpf_if->bif_dlist o kern/158726 net [ip6] [patch] ICMPv6 Router Announcement flooding limi o kern/158694 net [ix] [lagg] ix0 is not working within lagg(4) o kern/158665 net [ip6] [panic] kernel pagefault in in6_setscope() o kern/158635 net [em] TSO breaks BPF packet captures with em driver f kern/157802 net [dummynet] [panic] kernel panic in dummynet o kern/157785 net amd64 + jail + ipfw + natd = very slow outbound traffi o kern/157418 net [em] em driver lockup during boot on Supermicro X9SCM- o kern/157410 net [ip6] IPv6 Router Advertisements Cause Excessive CPU U o kern/157287 net [re] [panic] INVARIANTS panic (Memory modified after f o kern/157200 net [network.subr] [patch] stf(4) can not communicate betw o kern/157182 net [lagg] lagg interface not working together with epair o kern/156877 net [dummynet] [panic] dummynet move_pkt() null ptr derefe o kern/156667 net [em] em0 fails to init on CURRENT after March 17 o kern/156408 net [vlan] Routing failure when using VLANs vs. Physical e o kern/156328 net [icmp]: host can ping other subnet but no have IP from o kern/156317 net [ip6] Wrong order of IPv6 NS DAD/MLD Report o kern/156279 net [if_bridge][divert][ipfw] unable to correctly re-injec o kern/156226 net [lagg]: failover does not announce the failover to swi o kern/156030 net [ip6] [panic] Crash in nd6_dad_start() due to null ptr o kern/155680 net [multicast] problems with multicast s kern/155642 net [new driver] [request] Add driver for Realtek RTL8191S o kern/155597 net [panic] Kernel panics with "sbdrop" message o kern/155420 net [vlan] adding vlan break existent vlan o kern/155177 net [route] [panic] Panic when inject routes in kernel o kern/155010 net [msk] ntfs-3g via iscsi using msk driver cause kernel o kern/154943 net [gif] ifconfig gifX create on existing gifX clears IP s kern/154851 net [new driver] [request]: Port brcm80211 driver from Lin o kern/154850 net [netgraph] [patch] ng_ether fails to name nodes when t o kern/154679 net [em] Fatal trap 12: "em1 taskq" only at startup (8.1-R o kern/154600 net [tcp] [panic] Random kernel panics on tcp_output o kern/154557 net [tcp] Freeze tcp-session of the clients, if in the gat o kern/154443 net [if_bridge] Kernel module bridgestp.ko missing after u o kern/154286 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/154255 net [nfs] NFS not responding o kern/154214 net [stf] [panic] Panic when creating stf interface o kern/154185 net race condition in mb_dupcl p kern/154169 net [multicast] [ip6] Node Information Query multicast add o kern/154134 net [ip6] stuck kernel state in LISTEN on ipv6 daemon whic o kern/154091 net [netgraph] [panic] netgraph, unaligned mbuf? o conf/154062 net [vlan] [patch] change to way of auto-generatation of v o kern/153937 net [ral] ralink panics the system (amd64 freeBSDD 8.X) wh o kern/153936 net [ixgbe] [patch] MPRC workaround incorrectly applied to o kern/153816 net [ixgbe] ixgbe doesn't work properly with the Intel 10g o kern/153772 net [ixgbe] [patch] sysctls reference wrong XON/XOFF varia o kern/153497 net [netgraph] netgraph panic due to race conditions o kern/153454 net [patch] [wlan] [urtw] Support ad-hoc and hostap modes o kern/153308 net [em] em interface use 100% cpu o kern/153244 net [em] em(4) fails to send UDP to port 0xffff o kern/152893 net [netgraph] [panic] 8.2-PRERELEASE panic in netgraph o kern/152853 net [em] tftpd (and likely other udp traffic) fails over e o kern/152828 net [em] poor performance on 8.1, 8.2-PRE o kern/152569 net [net]: Multiple ppp connections and routing table prob o kern/152235 net [arp] Permanent local ARP entries are not properly upd o kern/152141 net [vlan] [patch] encapsulate vlan in ng_ether before out o kern/152036 net [libc] getifaddrs(3) returns truncated sockaddrs for n o kern/151690 net [ep] network connectivity won't work until dhclient is o kern/151681 net [nfs] NFS mount via IPv6 leads to hang on client with o kern/151593 net [igb] [panic] Kernel panic when bringing up igb networ o kern/150920 net [ixgbe][igb] Panic when packets are dropped with heade o kern/150557 net [igb] igb0: Watchdog timeout -- resetting o kern/150251 net [patch] [ixgbe] Late cable insertion broken o kern/150249 net [ixgbe] Media type detection broken o bin/150224 net ppp(8) does not reassign static IP after kill -KILL co f kern/149969 net [wlan] [ral] ralink rt2661 fails to maintain connectio o kern/149643 net [rum] device not sending proper beacon frames in ap mo o kern/149609 net [panic] reboot after adding second default route o kern/149117 net [inet] [patch] in_pcbbind: redundant test o kern/149086 net [multicast] Generic multicast join failure in 8.1 o kern/148018 net [flowtable] flowtable crashes on ia64 o kern/147912 net [boot] FreeBSD 8 Beta won't boot on Thinkpad i1300 11 o kern/147894 net [ipsec] IPv6-in-IPv4 does not work inside an ESP-only o kern/147155 net [ip6] setfb not work with ipv6 o kern/146845 net [libc] close(2) returns error 54 (connection reset by f kern/146792 net [flowtable] flowcleaner 100% cpu's core load o kern/146719 net [pf] [panic] PF or dumynet kernel panic o kern/146534 net [icmp6] wrong source address in echo reply o kern/146427 net [mwl] Additional virtual access points don't work on m f kern/146394 net [vlan] IP source address for outgoing connections o bin/146377 net [ppp] [tun] Interface doesn't clear addresses when PPP o kern/146358 net [vlan] wrong destination MAC address o kern/146165 net [wlan] [panic] Setting bssid in adhoc mode causes pani o kern/146082 net [ng_l2tp] a false invaliant check was performed in ng_ o kern/146037 net [panic] mpd + CoA = kernel panic o kern/145825 net [panic] panic: soabort: so_count o kern/145728 net [lagg] Stops working lagg between two servers. p kern/145600 net TCP/ECN behaves different to CE/CWR than ns2 reference f kern/144917 net [flowtable] [panic] flowtable crashes system [regressi o kern/144882 net MacBookPro =>4.1 does not connect to BSD in hostap wit o kern/144874 net [if_bridge] [patch] if_bridge frees mbuf after pfil ho o conf/144700 net [rc.d] async dhclient breaks stuff for too many people o kern/144616 net [nat] [panic] ip_nat panic FreeBSD 7.2 f kern/144315 net [ipfw] [panic] freebsd 8-stable reboot after add ipfw o kern/144231 net bind/connect/sendto too strict about sockaddr length o kern/143846 net [gif] bringing gif3 tunnel down causes gif0 tunnel to s kern/143673 net [stf] [request] there should be a way to support multi o kern/143622 net [pfil] [patch] unlock pfil lock while calling firewall o kern/143593 net [ipsec] When using IPSec, tcpdump doesn't show outgoin o kern/143591 net [ral] RT2561C-based DLink card (DWL-510) fails to work o kern/143208 net [ipsec] [gif] IPSec over gif interface not working o kern/143034 net [panic] system reboots itself in tcp code [regression] o kern/142877 net [hang] network-related repeatable 8.0-STABLE hard hang o kern/142774 net Problem with outgoing connections on interface with mu o kern/142772 net [libc] lla_lookup: new lle malloc failed f kern/142518 net [em] [lagg] Problem on 8.0-STABLE with em and lagg o kern/142018 net [iwi] [patch] Possibly wrong interpretation of beacon- o kern/141861 net [wi] data garbled with WEP and wi(4) with Prism 2.5 f kern/141741 net Etherlink III NIC won't work after upgrade to FBSD 8, o kern/140742 net rum(4) Two asus-WL167G adapters cannot talk to each ot o kern/140682 net [netgraph] [panic] random panic in netgraph f kern/140634 net [vlan] destroying if_lagg interface with if_vlan membe o kern/140619 net [ifnet] [patch] refine obsolete if_var.h comments desc o kern/140346 net [wlan] High bandwidth use causes loss of wlan connecti o kern/140142 net [ip6] [panic] FreeBSD 7.2-amd64 panic w/IPv6 o kern/140066 net [bwi] install report for 8.0 RC 2 (multiple problems) o kern/139387 net [ipsec] Wrong lenth of PF_KEY messages in promiscuous o bin/139346 net [patch] arp(8) add option to remove static entries lis o kern/139268 net [if_bridge] [patch] allow if_bridge to forward just VL p kern/139204 net [arp] DHCP server replies rejected, ARP entry lost bef o kern/139117 net [lagg] + wlan boot timing (EBUSY) o kern/138850 net [dummynet] dummynet doesn't work correctly on a bridge o kern/138782 net [panic] sbflush_internal: cc 0 || mb 0xffffff004127b00 o kern/138688 net [rum] possibly broken on 8 Beta 4 amd64: able to wpa a o kern/138678 net [lo] FreeBSD does not assign linklocal address to loop o kern/138407 net [gre] gre(4) interface does not come up after reboot o kern/138332 net [tun] [lor] ifconfig tun0 destroy causes LOR if_adata/ o kern/138266 net [panic] kernel panic when udp benchmark test used as r f kern/138029 net [bpf] [panic] periodically kernel panic and reboot o kern/137881 net [netgraph] [panic] ng_pppoe fatal trap 12 p bin/137841 net [patch] wpa_supplicant(8) cannot verify SHA256 signed p kern/137776 net [rum] panic in rum(4) driver on 8.0-BETA2 o bin/137641 net ifconfig(8): various problems with "vlan_device.vlan_i o kern/137392 net [ip] [panic] crash in ip_nat.c line 2577 o kern/137372 net [ral] FreeBSD doesn't support wireless interface from o kern/137089 net [lagg] lagg falsely triggers IPv6 duplicate address de o kern/136911 net [netgraph] [panic] system panic on kldload ng_bpf.ko t o kern/136618 net [pf][stf] panic on cloning interface without unit numb o kern/135502 net [periodic] Warning message raised by rtfree function i o kern/134583 net [hang] Machine with jail freezes after random amount o o kern/134531 net [route] [panic] kernel crash related to routes/zebra o kern/134157 net [dummynet] dummynet loads cpu for 100% and make a syst o kern/133969 net [dummynet] [panic] Fatal trap 12: page fault while in o kern/133968 net [dummynet] [panic] dummynet kernel panic o kern/133736 net [udp] ip_id not protected ... o kern/133595 net [panic] Kernel Panic at pcpu.h:195 o kern/133572 net [ppp] [hang] incoming PPTP connection hangs the system o kern/133490 net [bpf] [panic] 'kmem_map too small' panic on Dell r900 o kern/133235 net [netinet] [patch] Process SIOCDLIFADDR command incorre f kern/133213 net arp and sshd errors on 7.1-PRERELEASE o kern/133060 net [ipsec] [pfsync] [panic] Kernel panic with ipsec + pfs o kern/132889 net [ndis] [panic] NDIS kernel crash on load BCM4321 AGN d o conf/132851 net [patch] rc.conf(5): allow to setfib(1) for service run o kern/132734 net [ifmib] [panic] panic in net/if_mib.c o kern/132705 net [libwrap] [patch] libwrap - infinite loop if hosts.all o kern/132672 net [ndis] [panic] ndis with rt2860.sys causes kernel pani o kern/132354 net [nat] Getting some packages to ipnat(8) causes crash o kern/132277 net [crypto] [ipsec] poor performance using cryptodevice f o kern/131781 net [ndis] ndis keeps dropping the link o kern/131776 net [wi] driver fails to init o kern/131753 net [altq] [panic] kernel panic in hfsc_dequeue o bin/131365 net route(8): route add changes interpretation of network f kern/130820 net [ndis] wpa_supplicant(8) returns 'no space on device' o kern/130628 net [nfs] NFS / rpc.lockd deadlock on 7.1-R o kern/130525 net [ndis] [panic] 64 bit ar5008 ndisgen-erated driver cau o kern/130311 net [wlan_xauth] [panic] hostapd restart causing kernel pa o kern/130109 net [ipfw] Can not set fib for packets originated from loc f kern/130059 net [panic] Leaking 50k mbufs/hour f kern/129719 net [nfs] [panic] Panic during shutdown, tcp_ctloutput: in o kern/129517 net [ipsec] [panic] double fault / stack overflow f kern/129508 net [carp] [panic] Kernel panic with EtherIP (may be relat o kern/129219 net [ppp] Kernel panic when using kernel mode ppp o kern/129197 net [panic] 7.0 IP stack related panic o bin/128954 net ifconfig(8) deletes valid routes o bin/128602 net [an] wpa_supplicant(8) crashes with an(4) o kern/128448 net [nfs] 6.4-RC1 Boot Fails if NFS Hostname cannot be res o bin/128295 net [patch] ifconfig(8) does not print TOE4 or TOE6 capabi o bin/128001 net wpa_supplicant(8), wlan(4), and wi(4) issues o kern/127826 net [iwi] iwi0 driver has reduced performance and connecti o kern/127815 net [gif] [patch] if_gif does not set vlan attributes from o kern/127724 net [rtalloc] rtfree: 0xc5a8f870 has 1 refs f bin/127719 net [arp] arp: Segmentation fault (core dumped) f kern/127528 net [icmp]: icmp socket receives icmp replies not owned by p kern/127360 net [socket] TOE socket options missing from sosetopt() o bin/127192 net routed(8) removes the secondary alias IP of interface f kern/127145 net [wi]: prism (wi) driver crash at bigger traffic o kern/126895 net [patch] [ral] Add antenna selection (marked as TBD) o kern/126874 net [vlan]: Zebra problem if ifconfig vlanX destroy o kern/126695 net rtfree messages and network disruption upon use of if_ o kern/126339 net [ipw] ipw driver drops the connection o kern/126075 net [inet] [patch] internet control accesses beyond end of o bin/125922 net [patch] Deadlock in arp(8) o kern/125920 net [arp] Kernel Routing Table loses Ethernet Link status o kern/125845 net [netinet] [patch] tcp_lro_rx() should make use of hard o kern/125258 net [socket] socket's SO_REUSEADDR option does not work o kern/125239 net [gre] kernel crash when using gre o kern/124341 net [ral] promiscuous mode for wireless device ral0 looses o kern/124225 net [ndis] [patch] ndis network driver sometimes loses net o kern/124160 net [libc] connect(2) function loops indefinitely o kern/124021 net [ip6] [panic] page fault in nd6_output() o kern/123968 net [rum] [panic] rum driver causes kernel panic with WPA. o kern/123892 net [tap] [patch] No buffer space available o kern/123890 net [ppp] [panic] crash & reboot on work with PPP low-spee o kern/123858 net [stf] [patch] stf not usable behind a NAT o kern/123758 net [panic] panic while restarting net/freenet6 o bin/123633 net ifconfig(8) doesn't set inet and ether address in one o kern/123559 net [iwi] iwi periodically disassociates/associates [regre o bin/123465 net [ip6] route(8): route add -inet6 -interfac o kern/123463 net [ipsec] [panic] repeatable crash related to ipsec-tool o conf/123330 net [nsswitch.conf] Enabling samba wins in nsswitch.conf c o kern/123160 net [ip] Panic and reboot at sysctl kern.polling.enable=0 o kern/122989 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/122954 net [lagg] IPv6 EUI64 incorrectly chosen for lagg devices f kern/122780 net [lagg] tcpdump on lagg interface during high pps wedge o kern/122685 net It is not visible passing packets in tcpdump(1) o kern/122319 net [wi] imposible to enable ad-hoc demo mode with Orinoco o kern/122290 net [netgraph] [panic] Netgraph related "kmem_map too smal o kern/122252 net [ipmi] [bge] IPMI problem with BCM5704 (does not work o kern/122033 net [ral] [lor] Lock order reversal in ral0 at bootup ieee o bin/121895 net [patch] rtsol(8)/rtsold(8) doesn't handle managed netw s kern/121774 net [swi] [panic] 6.3 kernel panic in swi1: net o kern/121555 net [panic] Fatal trap 12: current process = 12 (swi1: net o kern/121534 net [ipl] [nat] FreeBSD Release 6.3 Kernel Trap 12: o kern/121443 net [gif] [lor] icmp6_input/nd6_lookup o kern/121437 net [vlan] Routing to layer-2 address does not work on VLA o bin/121359 net [patch] [security] ppp(8): fix local stack overflow in o kern/121257 net [tcp] TSO + natd -> slow outgoing tcp traffic o kern/121181 net [panic] Fatal trap 3: breakpoint instruction fault whi o kern/120966 net [rum] kernel panic with if_rum and WPA encryption o kern/120566 net [request]: ifconfig(8) make order of arguments more fr o kern/120304 net [netgraph] [patch] netgraph source assumes 32-bit time o kern/120266 net [udp] [panic] gnugk causes kernel panic when closing U o bin/120060 net routed(8) deletes link-level routes in the presence of o kern/119945 net [rum] [panic] rum device in hostap mode, cause kernel o kern/119791 net [nfs] UDP NFS mount of aliased IP addresses from a Sol o kern/119617 net [nfs] nfs error on wpa network when reseting/shutdown f kern/119516 net [ip6] [panic] _mtx_lock_sleep: recursed on non-recursi o kern/119432 net [arp] route add -host -iface causes arp e o kern/119225 net [wi] 7.0-RC1 no carrier with Prism 2.5 wifi card [regr o kern/118727 net [netgraph] [patch] [request] add new ng_pf module o kern/117423 net [vlan] Duplicate IP on different interfaces o bin/117339 net [patch] route(8): loading routing management commands o bin/116643 net [patch] [request] fstat(1): add INET/INET6 socket deta o kern/116185 net [iwi] if_iwi driver leads system to reboot o kern/115239 net [ipnat] panic with 'kmem_map too small' using ipnat o kern/115019 net [netgraph] ng_ether upper hook packet flow stops on ad o kern/115002 net [wi] if_wi timeout. failed allocation (busy bit). ifco o kern/114915 net [patch] [pcn] pcn (sys/pci/if_pcn.c) ethernet driver f o kern/113432 net [ucom] WARNING: attempt to net_add_domain(netgraph) af o kern/112722 net [ipsec] [udp] IP v4 udp fragmented packet reject o kern/112686 net [patm] patm driver freezes System (FreeBSD 6.2-p4) i38 o bin/112557 net [patch] ppp(8) lock file should not use symlink name o kern/112528 net [nfs] NFS over TCP under load hangs with "impossible p o kern/111537 net [inet6] [patch] ip6_input() treats mbuf cluster wrong o kern/111457 net [ral] ral(4) freeze o kern/110284 net [if_ethersubr] Invalid Assumption in SIOCSIFADDR in et o kern/110249 net [kernel] [regression] [patch] setsockopt() error regre o kern/109470 net [wi] Orinoco Classic Gold PC Card Can't Channel Hop o bin/108895 net pppd(8): PPPoE dead connections on 6.2 [regression] f kern/108197 net [panic] [gif] [ip6] if_delmulti reference counting pan o kern/107944 net [wi] [patch] Forget to unlock mutex-locks o conf/107035 net [patch] bridge(8): bridge interface given in rc.conf n o kern/106444 net [netgraph] [panic] Kernel Panic on Binding to an ip to o kern/106316 net [dummynet] dummynet with multipass ipfw drops packets o kern/105945 net Address can disappear from network interface s kern/105943 net Network stack may modify read-only mbuf chain copies o bin/105925 net problems with ifconfig(8) and vlan(4) [regression] o kern/104851 net [inet6] [patch] On link routes not configured when usi o kern/104751 net [netgraph] kernel panic, when getting info about my tr o kern/104738 net [inet] [patch] Reentrant problem with inet_ntoa in the o kern/103191 net Unpredictable reboot o kern/103135 net [ipsec] ipsec with ipfw divert (not NAT) encodes a pac o kern/102540 net [netgraph] [patch] supporting vlan(4) by ng_fec(4) o conf/102502 net [netgraph] [patch] ifconfig name does't rename netgrap o kern/102035 net [plip] plip networking disables parallel port printing o kern/100709 net [libc] getaddrinfo(3) should return TTL info o kern/100519 net [netisr] suggestion to fix suboptimal network polling o kern/98597 net [inet6] Bug in FreeBSD 6.1 IPv6 link-local DAD procedu o bin/98218 net wpa_supplicant(8) blacklist not working o kern/97306 net [netgraph] NG_L2TP locks after connection with failed o conf/97014 net [gif] gifconfig_gif? in rc.conf does not recognize IPv f kern/96268 net [socket] TCP socket performance drops by 3000% if pack o kern/95519 net [ral] ral0 could not map mbuf o kern/95288 net [pppd] [tty] [panic] if_ppp panic in sys/kern/tty_subr o kern/95277 net [netinet] [patch] IP Encapsulation mask_match() return o kern/95267 net packet drops periodically appear f kern/93378 net [tcp] Slow data transfer in Postfix and Cyrus IMAP (wo o kern/93019 net [ppp] ppp and tunX problems: no traffic after restarti o kern/92880 net [libc] [patch] almost rewritten inet_network(3) functi s kern/92279 net [dc] Core faults everytime I reboot, possible NIC issu o kern/91859 net [ndis] if_ndis does not work with Asus WL-138 o kern/91364 net [ral] [wep] WF-511 RT2500 Card PCI and WEP o kern/91311 net [aue] aue interface hanging o kern/87421 net [netgraph] [panic]: ng_ether + ng_eiface + if_bridge o kern/86871 net [tcp] [patch] allocation logic for PCBs in TIME_WAIT s o kern/86427 net [lor] Deadlock with FASTIPSEC and nat o kern/85780 net 'panic: bogus refcnt 0' in routing/ipv6 o bin/85445 net ifconfig(8): deprecated keyword to ifconfig inoperativ o bin/82975 net route change does not parse classfull network as given o kern/82881 net [netgraph] [panic] ng_fec(4) causes kernel panic after o kern/82468 net Using 64MB tcp send/recv buffers, trafficflow stops, i o bin/82185 net [patch] ndp(8) can delete the incorrect entry o kern/81095 net IPsec connection stops working if associated network i o kern/78968 net FreeBSD freezes on mbufs exhaustion (network interface o kern/78090 net [ipf] ipf filtering on bridged packets doesn't work if o kern/77341 net [ip6] problems with IPV6 implementation o kern/75873 net Usability problem with non-RFC-compliant IP spoof prot s kern/75407 net [an] an(4): no carrier after short time a kern/71474 net [route] route lookup does not skip interfaces marked d o kern/71469 net default route to internet magically disappears with mu o kern/68889 net [panic] m_copym, length > size of mbuf chain o kern/66225 net [netgraph] [patch] extend ng_eiface(4) control message o kern/65616 net IPSEC can't detunnel GRE packets after real ESP encryp s kern/60293 net [patch] FreeBSD arp poison patch a kern/56233 net IPsec tunnel (ESP) over IPv6: MTU computation is wrong s bin/41647 net ifconfig(8) doesn't accept lladdr along with inet addr o kern/39937 net ipstealth issue a kern/38554 net [patch] changing interface ipaddress doesn't seem to w o kern/31940 net ip queue length too short for >500kpps o kern/31647 net [libc] socket calls can return undocumented EINVAL o kern/30186 net [libc] getaddrinfo(3) does not handle incorrect servna f kern/24959 net [patch] proper TCP_NOPUSH/TCP_CORK compatibility o conf/23063 net [arp] [patch] for static ARP tables in rc.network o kern/21998 net [socket] [patch] ident only for outgoing connections o kern/5877 net [socket] sb_cc counts control data as well as data dat 472 problems total. From owner-freebsd-net@FreeBSD.ORG Mon Dec 16 14:35:10 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A8021A5E; Mon, 16 Dec 2013 14:35:10 +0000 (UTC) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7AA8C14CD; Mon, 16 Dec 2013 14:35:10 +0000 (UTC) Received: from compute5.internal (compute5.nyi.mail.srv.osa [10.202.2.45]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 97A6420E05; Mon, 16 Dec 2013 09:35:00 -0500 (EST) Received: from web3 ([10.202.2.213]) by compute5.internal (MEProxy); Mon, 16 Dec 2013 09:35:00 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:mime-version :content-transfer-encoding:content-type:subject:date; s=smtpout; bh=A9n63p96CGowysF88oow4a/swNw=; b=hVw1JTEDnmWJTSFE4F7XFkulpYms HxACgrzQqal94kissY/gcxWGTkA/eB930FU9bwaRXJ09dTzBNu053Oxh8aBZqgVw tqWQ3LI38XF8Sjr5bHGJPIjmjUaf0MGOUGcgwYxBDkUIwJqe6RdogC/sbTfd309o GAN+qLsQnIw63lw= Received: by web3.nyi.mail.srv.osa (Postfix, from userid 99) id 52C83115EE8; Mon, 16 Dec 2013 09:35:00 -0500 (EST) Message-Id: <1387204500.12061.60192349.19EAE1B4@webmail.messagingengine.com> X-Sasl-Enc: 6jYsIDzJXFCcPwwYCIHUdBMQroS8kAtvrkoR0P5S3+Ga 1387204500 From: Mark Felder To: freebsd-net@freebsd.org, freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-7f24e732 Subject: 10.0-RC1: bad mbuf leak? Date: Mon, 16 Dec 2013 08:35:00 -0600 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 14:35:10 -0000 Hi all, I think I'm experiencing a bad mbuf leak or something of the sort and I don't know how to diagnose this further. I have a machine at home that is mostly used for transcoding video for viewing on my TV via the multimedia/plexmediaserver port. This software runs in a jail and gets the actual files from my NAS via NFSv4. It's a pretty simple setup and sits idle unless I am watching TV. Between the 10.0-BETAs and the 10.0-RC1 did something network related that could affect mbufs change? Ever since I upgraded this machine to RC1 it has been "crashing", which I diagnosed as actually being an mbuf exhaustion. Raising the mbufs brings it back to life, and it does mention the exhaustion on the system console. Last night, for example, I rebooted the machine and it has been sitting mostly idle. I wake up this morning to see this: # vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP mbuf_packet: 256, 6511095, 1023, 1727, 8322474, 0, 0 mbuf: 256, 6511095, 2811247, 1563,56000603,121933, 0 mbuf_cluster: 2048, 1017358, 2750, 0, 2750,2740, 0 mbuf_jumbo_page: 4096, 508679, 0, 152, 2831466, 137, 0 # netstat -m 812270/3290/2815560 mbufs in use (current/cache/total) 1023/1727/2750/1017358 mbuf clusters in use (current/cache/total/max) 1023/1727 mbuf+clusters out of packet secondary zone in use (current/cache) 0/152/152/508679 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/150719 9k jumbo clusters in use (current/cache/total/max) 0/0/0/84779 16k jumbo clusters in use (current/cache/total/max) 705113K/4884K/709998K bytes allocated to network (current/cache/total) 121933/2740/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 137/0/0 requests for jumbo clusters denied (4k/9k/16k) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile The network interface is em(4). Things I've tried: - restarting all software/services including the jail - down/up the network interface The only thing that works is rebooting. Also, the only possible "strange" part of this setup is that the NFS mounts used by the jail are not direct. They're actually nullfs mounted into the jail as I want access to them outside of the jail as well. Not sure if nullfs+nfs could do something this bizarre. If anyone has any hints on what I can do to track this down it would be appreciated. From owner-freebsd-net@FreeBSD.ORG Mon Dec 16 19:06:16 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DB35B442 for ; Mon, 16 Dec 2013 19:06:15 +0000 (UTC) Received: from mail-qe0-x22a.google.com (mail-qe0-x22a.google.com [IPv6:2607:f8b0:400d:c02::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 989811059 for ; Mon, 16 Dec 2013 19:06:15 +0000 (UTC) Received: by mail-qe0-f42.google.com with SMTP id b4so4329881qen.1 for ; Mon, 16 Dec 2013 11:06:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:sender:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=tKW5P/qEz1KpNZUIMWP7hCr5rxpChRhwFtbvhRrUueY=; b=aDQNbLYntK+rOpnU/VTW73hFvhbDCOUC0LMIJkVF5buelG8e1zKV8fGGW6yYZsyPZw y+K1tEQ88oeJs7ToBPMBlXCqaiL+2gv+ZRBEyiXjl7rlpa7rTQDbdcdohBQhaZQ0OojT +8DsuPrHQm8FPNknRePEObHYn594YWhY1F3ndjpdbhOJxv7FUXlKHOlXLE+18CEgwFaE Y+BbzBQzfL9o7ueUJHY38OMPUlN6N9KLed1j2VLO5SZlNwtFP1OCMpeuFKtiWXPGQq2R 98tTRbiBY1SoO5xelU+aoixe0KM1z1D2Vsvt0yCr35SCi8oMzN68uXYFYjWoWMgeuXG+ Ao1w== MIME-Version: 1.0 X-Received: by 10.224.160.210 with SMTP id o18mr34365748qax.57.1387220774854; Mon, 16 Dec 2013 11:06:14 -0800 (PST) Sender: shteryana@gmail.com Received: by 10.224.128.72 with HTTP; Mon, 16 Dec 2013 11:06:14 -0800 (PST) In-Reply-To: <20131216014704.GB3345@michelle.cdnetworks.com> References: <20131216014704.GB3345@michelle.cdnetworks.com> Date: Mon, 16 Dec 2013 21:06:14 +0200 X-Google-Sender-Auth: 01DwPjqLTYic7FFAinJJ1vBgRaM Message-ID: Subject: Re: RTL8111/8168B NIC & FreeBSD From: Shteryana Shopova To: pyunyh@gmail.com Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@FreeBSD.org" , David Rufino X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: syrinx@FreeBSD.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 19:06:16 -0000 Hm, It's neither - re0: port 0x2000-0x20ff mem 0xd1404000-0xd1404fff,0xd1400000-0xd1403fff irq 18 at device 0.1 on pci4 re0: Using 1 MSI-X message re0: ASPM disabled re0: Chip rev. 0x5c800000 re0: MAC rev. 0x00000000 cheers, Shteryana On Mon, Dec 16, 2013 at 3:47 AM, Yonghyeon PYUN wrote: > On Sun, Dec 15, 2013 at 06:46:35AM +0200, Shteryana Shopova wrote: >> Hi, >> >> I hit the same problem with the same chip on FreeBSD 11.0-CURRENT #0 >> r258178 - the card would receive nothing or (perhaps not-so-)random >> crap. ifconfig down/up and disabling RX/TX check-summing makes the >> card work for me as a temporary workaround. >> > > David's controller is RTL8168G. Are you sure you have the same > controller? If your controller is RTL8168E-VL(Chip rev. > 0x2c800000 in dmesg), try the patch at the following URL. > http://people.freebsd.org/~yongari/re/re.8168evl.diff > >> re0@pci0:4:0:1: class=0x020000 card=0x07981025 chip=0x816810ec >> rev=0x14 hdr=0x00 >> vendor = 'Realtek Semiconductor Co., Ltd.' >> device = 'RTL8111/8168B PCI Express Gigabit Ethernet controller' >> class = network >> subclass = ethernet >> >> cheers, >> Shteryana >> >> >> On Sat, Dec 14, 2013 at 11:31 PM, David Rufino wrote: >> > Hello, >> > >> > I'm having similar problems using an onboard RTL 8168B NIC. I noticed >> > there were some recent changes to the re0 driver, so I tried -STABLE >> > (r258961). The device is detected but can't send or receive packets. >> > Interestingly it doesn't work in Debian wheezy, but does work fine in >> > Windows 7 with latest drivers. I'm happy to assist in any way. Here's >> > some relevant debugging information >> > >> > pciconf -l -v >> > >> > re0@pci0:1:0:0: class=0x020000 card=0x81161019 chip=0x816810ec rev=0x0c hdr=0x00 >> > vendor = 'Realtek Semiconductor Co., Ltd.' >> > device = 'RTL8111/8168B PCI Express Gigabit Ethernet controller' >> > class = network >> > subclass = ethernet >> > >> > re0: port >> > 0xe000-0xe0ff mem 0xfea00000-0xfea00fff,0xd0800000-0xd0803fff irq 32 >> > at device 0.0 on pci1 >> > re0: Using 1 MSI-X message >> > re0: ASPM disabled >> > re0: Chip rev. 0x4c000000 >> > re0: MAC rev. 0x00000000 >> > miibus0: on re0 >> > re0: Ethernet address: 74:27:ea:d3:de:5a >> > >> > Thanks!, >> > David >> > >> >> >> >> Hello, >> >> >> >> we are trying to install FreeBSD on a computer that uses the NIC mentioned >> >> above. The NIC is running under linux without problems, which we've tested >> >> for several days transferring several GB of data. From owner-freebsd-net@FreeBSD.ORG Mon Dec 16 19:36:17 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D6CA1E8A; Mon, 16 Dec 2013 19:36:17 +0000 (UTC) Received: from mail-qa0-x22c.google.com (mail-qa0-x22c.google.com [IPv6:2607:f8b0:400d:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 701DF1254; Mon, 16 Dec 2013 19:36:17 +0000 (UTC) Received: by mail-qa0-f44.google.com with SMTP id i13so1856820qae.3 for ; Mon, 16 Dec 2013 11:36:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=mRaKouMkvKsXYTqoyj8k1CRI51SaerxJ/jYGfjVAYso=; b=MlnlOCA0+UT3t3fX9GrwKP58+wsUORi7ndkqnhFTAJK0VlavSiNAFo8tMgl2Xn9xxp Dpa4XpJXS7+xe8odw8VYBavsWKWT27njcCNU0cpxAh4PRQMrSic1z272eT1tmBk5le+K EH1aAxCXUVmKxoRDalF0vtmd75JxfKioZCbuI9/ewh20nWSVXVMdSeq6BSl3Z44KaLTb gGsf1kyollQ04hdZNSjJZEgSxGVUXKoUavV+0nNjRNM1WRKp2eWaliBqHTxMHwyjlXu9 GtfNhZ9OkHfvFc6JdDn/UchR/DfbHLGLR03c+hQzeOvPCbwn9WxovLmXHLBpfgGm0zt+ p8vQ== MIME-Version: 1.0 X-Received: by 10.224.50.195 with SMTP id a3mr35153412qag.25.1387222576570; Mon, 16 Dec 2013 11:36:16 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Mon, 16 Dec 2013 11:36:16 -0800 (PST) In-Reply-To: <201312131717.10863.jhb@freebsd.org> References: <521B9C2A-EECC-4412-9F68-2235320EF324@lurchi.franken.de> <201312131326.28952.jhb@freebsd.org> <201312131717.10863.jhb@freebsd.org> Date: Mon, 16 Dec 2013 11:36:16 -0800 X-Google-Sender-Auth: Wh5mvbYl0NKsjfgseE3VOGbC3Yg Message-ID: Subject: Re: A small fix for if_em.c, if_igb.c, if_ixgbe.c From: Adrian Chadd To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: Yong-Hyeon Pyun , FreeBSD Net , John-Mark Gurney , Jack F Vogel , Michael Tuexen X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 19:36:17 -0000 On 13 December 2013 14:17, John Baldwin wrote: >> Anyone using if_start() failing as "the frame i just queued failed to >> transmit" is broken and well, we should just replace it with >> if_transmit(). > > Hmm, I was a bit wrong. Driver if_start routines return void, so they > only failed if the IFQ filled up completely. In that case, I think it is > fine to move forward with what you want (only return an error for failures > involving the packet passed to if_transmit), but I don't think we should > hange if_transmit drivers to just always return 0. i agree. if_transmit() should return 0 only if: * the driver queued it internally and intends to try transmitting it later; * the driver directly dispatched the frame to the hardware. If it failed to do either of the above, it should return an error. How's that sound? -a From owner-freebsd-net@FreeBSD.ORG Mon Dec 16 20:06:50 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2FDF0F0; Mon, 16 Dec 2013 20:06:50 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B42EB15E3; Mon, 16 Dec 2013 20:06:49 +0000 (UTC) Received: from [192.168.1.200] (p508F0515.dip0.t-ipconnect.de [80.143.5.21]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 440E81C0C0692; Mon, 16 Dec 2013 21:06:46 +0100 (CET) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) Subject: Re: A small fix for if_em.c, if_igb.c, if_ixgbe.c From: Michael Tuexen In-Reply-To: Date: Mon, 16 Dec 2013 21:06:46 +0100 Content-Transfer-Encoding: 7bit Message-Id: <0BC9D25E-639A-4305-A51A-222AE645152C@lurchi.franken.de> References: <521B9C2A-EECC-4412-9F68-2235320EF324@lurchi.franken.de> <201312131326.28952.jhb@freebsd.org> <201312131717.10863.jhb@freebsd.org> To: Adrian Chadd X-Mailer: Apple Mail (2.1510) Cc: Yong-Hyeon Pyun , FreeBSD Net , John-Mark Gurney , Jack F Vogel , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 20:06:50 -0000 On Dec 16, 2013, at 8:36 PM, Adrian Chadd wrote: > On 13 December 2013 14:17, John Baldwin wrote: > >>> Anyone using if_start() failing as "the frame i just queued failed to >>> transmit" is broken and well, we should just replace it with >>> if_transmit(). >> >> Hmm, I was a bit wrong. Driver if_start routines return void, so they >> only failed if the IFQ filled up completely. In that case, I think it is >> fine to move forward with what you want (only return an error for failures >> involving the packet passed to if_transmit), but I don't think we should >> hange if_transmit drivers to just always return 0. > > i agree. if_transmit() should return 0 only if: > > * the driver queued it internally and intends to try transmitting it later; > * the driver directly dispatched the frame to the hardware. > > If it failed to do either of the above, it should return an error. > > How's that sound? That sounds good. However, The transport layer is interested in the case where if_transmit() returns a non-zero value. Does your statement imply: if_transmit() returns a non-zero value only if the packet will not make it on the wire (for example, it failed to queue it). Best regards Michael > > > -a > From owner-freebsd-net@FreeBSD.ORG Mon Dec 16 20:15:42 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 92A2746B; Mon, 16 Dec 2013 20:15:42 +0000 (UTC) Received: from mail-qe0-x236.google.com (mail-qe0-x236.google.com [IPv6:2607:f8b0:400d:c02::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 29A1F16C2; Mon, 16 Dec 2013 20:15:42 +0000 (UTC) Received: by mail-qe0-f54.google.com with SMTP id cy11so4406205qeb.27 for ; Mon, 16 Dec 2013 12:15:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=BYo/O6nsZMrOTzP3jOXp9XItoTBPo/gf0Owk5IxeTH8=; b=E2f0GglbEz54mlNRFd2cdZ1Ny65d63+CdoMcbizXeaFJpBnnLbiDXxjqcVqGLlBqbL EmW8ez4s57yVqi2qDTf2sVc9i5h7hKTlbUhJntAJQhOJYiJVGJCjgy4D5nuqWTvWKlXQ xziAf9me8qjkfxSK9Myjwiz65ainZ7xGLKNAv+yxEwsILPUgJZPdM4WEkgaA/6rCHzij pe1SAxSBP8qX/Gv2mJb6ZSTCfegabtbrqVPFgnGN6RFwzQZ9mZ38AkN8UHHWNhjv8eWW iXyBRBebALI/cg/fygbwyseF1oNHFGSjA1St6Q9MOF+hKJ9zKXUIoAdrjcVKJKVWJYCv EbAg== MIME-Version: 1.0 X-Received: by 10.224.89.73 with SMTP id d9mr36541909qam.5.1387224941296; Mon, 16 Dec 2013 12:15:41 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Mon, 16 Dec 2013 12:15:41 -0800 (PST) In-Reply-To: <0BC9D25E-639A-4305-A51A-222AE645152C@lurchi.franken.de> References: <521B9C2A-EECC-4412-9F68-2235320EF324@lurchi.franken.de> <201312131326.28952.jhb@freebsd.org> <201312131717.10863.jhb@freebsd.org> <0BC9D25E-639A-4305-A51A-222AE645152C@lurchi.franken.de> Date: Mon, 16 Dec 2013 12:15:41 -0800 X-Google-Sender-Auth: 9JUMG5_e_s90ZzhisfRLV7vCyOI Message-ID: Subject: Re: A small fix for if_em.c, if_igb.c, if_ixgbe.c From: Adrian Chadd To: Michael Tuexen Content-Type: text/plain; charset=ISO-8859-1 Cc: Yong-Hyeon Pyun , FreeBSD Net , John-Mark Gurney , Jack F Vogel , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 20:15:42 -0000 On 16 December 2013 12:06, Michael Tuexen wrote: >> i agree. if_transmit() should return 0 only if: >> >> * the driver queued it internally and intends to try transmitting it later; >> * the driver directly dispatched the frame to the hardware. >> >> If it failed to do either of the above, it should return an error. >> >> How's that sound? > That sounds good. However, The transport layer is interested in the case > where if_transmit() returns a non-zero value. > Does your statement imply: > if_transmit() returns a non-zero value only if the packet will not > make it on the wire (for example, it failed to queue it). If there's a queuing layer in the middle then we can't know that for certain. If the driver can't transmit the frame (eg it fails because of collisions, for example) then again, we can't know that for certain. What we can only know is that it was either queued and may or may not make it on the wire, or it wasn't queued/transmitted and it definitely _won't_ make it on the wire. -a From owner-freebsd-net@FreeBSD.ORG Mon Dec 16 20:19:41 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 47266764 for ; Mon, 16 Dec 2013 20:19:41 +0000 (UTC) Received: from mail-qe0-x232.google.com (mail-qe0-x232.google.com [IPv6:2607:f8b0:400d:c02::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 05672173E for ; Mon, 16 Dec 2013 20:19:40 +0000 (UTC) Received: by mail-qe0-f50.google.com with SMTP id 1so4313606qec.23 for ; Mon, 16 Dec 2013 12:19:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=pjlskNzVPUdNO7f2upts2EcI42ICplwIYBKLVcfSK4g=; b=jof1bsiYcKu/t7yV0TlL1NP3kkTpaMhQMQH/8bRLYpw7+kLMbGjZLxu96PJjaBCMBH ByIAsgOPuPM5CQCNoc2Hll4zPnk6uhLxvKrB6xJTS/TaETz8KPg2aSTCTHJfYt3Jd9yS wgl2TP84UQcODEbmd+Ed9bFe2TDuSCJJopmgJ4htkureEL6JtJsQj/Xw45VhA8uNCPa6 4xAVhglOJZqnCCRbOcxZGdHVUKNGoqmJjlhLkT7lnrQsLguvZ9Ry3Eb8xy1XALSn0F/b lxMpmKUSX1UbxUvckGr9mAh6mSQumkeNJOYJbR15foCWjgnOkoHZS870e1oOsmroGRLR 8Jew== MIME-Version: 1.0 X-Received: by 10.49.129.38 with SMTP id nt6mr36083875qeb.78.1387225180246; Mon, 16 Dec 2013 12:19:40 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Mon, 16 Dec 2013 12:19:40 -0800 (PST) In-Reply-To: References: Date: Mon, 16 Dec 2013 12:19:40 -0800 X-Google-Sender-Auth: jH79qQIC6eiCH7_JjdIvFzBPOPg Message-ID: Subject: Re: buf_ring in HEAD is racy From: Adrian Chadd To: Ryan Stone Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 20:19:41 -0000 So I've done some digging. The TL;DR - we really should expose the drops count in the ifa_data field. But that would involve an ABI change and .. well, that will be full of drama. The non-TL:DR: * The send queue drops are in the ifmibdata structure, which includes a few other things. But that requires you use the MIB interface to pull things out, rather than getifaddrs(). * getifaddrs() doesn't contain the max sendq length or drops, so we can't report those without adding a MIB fetch for the relevant interface. * bsnmp (which we use at work) correctly populates output discards by checking the MIB for ifmd_snd_drops. So I'd rather we fix ixgbe and other drivers by updating the send queue drops counter if the frames are dropped. That way it's consistent with other things. We should then do some of these things: * add a send queue drops field to the ifdata/ifadata where appropriate and make sure it gets correctly populated; * teach netstat to use ifmib instead of getifaddrs(); * as part of killing the ifsnd queue stuff, we should also migrate that particular drops counter out from there and to a top-level counter in ifnet, like the rest. Comments? -a On 14 December 2013 16:06, Adrian Chadd wrote: > oh cool, you just did the output-drops thing I was about to code up. > We're missing those counters at work and the ops guys poked me about > it. > > I'll also give that a whirl locally and see about working with jack to > get it into -HEAD / MFC'ed to 10. > > Thanks! > > > -a > > > On 13 December 2013 21:04, Ryan Stone wrote: >> I am seeing spurious output packet drops that appear to be due to >> insufficient memory barriers in buf_ring. I believe that this is the >> scenario that I am seeing: >> >> 1) The buf_ring is empty, br_prod_head = br_cons_head = 0 >> 2) Thread 1 attempts to enqueue an mbuf on the buf_ring. It fetches >> br_prod_head (0) into a local variable called prod_head >> 3) Thread 2 enqueues an mbuf on the buf_ring. The sequence of events >> is essentially: >> >> Thread 2 claims an index in the ring and atomically sets br_prod_head (say to 1) >> Thread 2 sets br_ring[1] = mbuf; >> Thread 2 does a full memory barrier >> Thread 2 updates br_prod_tail to 1 >> >> 4) Thread 2 dequeues the packet from the buf_ring using the >> single-consumer interface. The sequence of events is essentialy: >> >> Thread 2 checks whether queue is empty (br_cons_head == br_prod_tail), >> this is false >> Thread 2 sets br_cons_head to 1 >> Thread 2 grabs the mbuf from br_ring[1] >> Thread 2 sets br_cons_tail to 1 >> >> 5) Thread 1, which is still attempting to enqueue an mbuf on the ring. >> fetches br_cons_tail (1) into a local variable called cons_tail. It >> sees cons_tail == 1 but prod_head == 0 and concludes that the ring is >> full and drops the packet (incrementing br_drops unatomically, I might >> add) >> >> >> I can reproduce several drops per minute by configuring the ixgbe >> driver to use only 1 queue and then sending traffic from concurrent 8 >> iperf processes. (You will need this hacky patch to even see the >> drops with netstat, though: >> http://people.freebsd.org/~rstone/patches/ixgbe_br_drops.diff) >> >> I am investigating fixing buf_ring by using acquire/release semantics >> rather than load/store barriers. However, I note that this will >> apparently be the second attempt to fix buf_ring, and I'm seriously >> questioning whether this is worth the effort compared to the >> simplicity of using a mutex. I'm not even convinced that a correct >> lockless implementation will even be a performance win, given the >> number of memory barriers that will apparently be necessary. >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Mon Dec 16 21:04:21 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 170E3E97; Mon, 16 Dec 2013 21:04:21 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9A09F1D6B; Mon, 16 Dec 2013 21:04:20 +0000 (UTC) Received: from [192.168.1.200] (p508F0515.dip0.t-ipconnect.de [80.143.5.21]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id B7EF01C0C0692; Mon, 16 Dec 2013 22:04:18 +0100 (CET) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) Subject: Re: A small fix for if_em.c, if_igb.c, if_ixgbe.c From: Michael Tuexen In-Reply-To: Date: Mon, 16 Dec 2013 22:04:18 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <521B9C2A-EECC-4412-9F68-2235320EF324@lurchi.franken.de> <201312131326.28952.jhb@freebsd.org> <201312131717.10863.jhb@freebsd.org> <0BC9D25E-639A-4305-A51A-222AE645152C@lurchi.franken.de> To: Adrian Chadd X-Mailer: Apple Mail (2.1510) Cc: Yong-Hyeon Pyun , FreeBSD Net , John-Mark Gurney , Jack F Vogel , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Dec 2013 21:04:21 -0000 On Dec 16, 2013, at 9:15 PM, Adrian Chadd wrote: > On 16 December 2013 12:06, Michael Tuexen > wrote: >=20 >>> i agree. if_transmit() should return 0 only if: >>>=20 >>> * the driver queued it internally and intends to try transmitting it = later; >>> * the driver directly dispatched the frame to the hardware. >>>=20 >>> If it failed to do either of the above, it should return an error. >>>=20 >>> How's that sound? >> That sounds good. However, The transport layer is interested in the = case >> where if_transmit() returns a non-zero value. >> Does your statement imply: >> if_transmit() returns a non-zero value only if the packet will not >> make it on the wire (for example, it failed to queue it). >=20 > If there's a queuing layer in the middle then we can't know that for > certain. If the driver can't transmit the frame (eg it fails because > of collisions, for example) then again, we can't know that for > certain. >=20 > What we can only know is that it was either queued and may or may not > make it on the wire, or it wasn't queued/transmitted and it definitely > _won't_ make it on the wire. Correct. And I'm only interested in the "it wasn't queued/transmitted and it definitely _won't_ make it on the wire." part. So I would need something like if_transmit() returns an error only if it wasn't queued/transmitted and it definitely _won't_ make it on the wire. Acceptable for you? Best regards Michael >=20 >=20 >=20 > -a >=20 From owner-freebsd-net@FreeBSD.ORG Tue Dec 17 03:25:29 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0FD18600; Tue, 17 Dec 2013 03:25:29 +0000 (UTC) Received: from mail-qc0-x22a.google.com (mail-qc0-x22a.google.com [IPv6:2607:f8b0:400d:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 99C1C1119; Tue, 17 Dec 2013 03:25:28 +0000 (UTC) Received: by mail-qc0-f170.google.com with SMTP id x13so4534633qcv.1 for ; Mon, 16 Dec 2013 19:25:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=PybMX1TbeJNCpk9/eN1Qz5WvIt8FXyNEWt5yDmxCi3U=; b=X6zZP9sIkk5hmZ3i103A6LzHcJcyfarW2jOs1rslJxWxto1pF6YV/KJycaNE5dhXQV nvov32DmnkpwyYezWzCWFeqUJ2kC81tg7zwK0KNjtFJ2AFB/I5BIGcDW/NejIqVcAwCO Asswj4TBNfn0ELR3oBTGbaibQF9p5UTVpxNAcRC2K2dQ9Aj5gglYC2MCF8srTTjwqWBq 2QeUB+ProKqW80AlPASJ8gk8guk1rzsErz43Q7v4lyEfCrypYUhgwl/nQqpeSF6vXgjd SynB8t0yw1pA85f5cjXfpMhjpd9Jxmc59WqwAgNRLvUab+66fz+yWowpo/O2XJ+wzJ4W L1Hw== MIME-Version: 1.0 X-Received: by 10.229.56.200 with SMTP id z8mr38579851qcg.1.1387250727824; Mon, 16 Dec 2013 19:25:27 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Mon, 16 Dec 2013 19:25:27 -0800 (PST) In-Reply-To: References: <521B9C2A-EECC-4412-9F68-2235320EF324@lurchi.franken.de> <201312131326.28952.jhb@freebsd.org> <201312131717.10863.jhb@freebsd.org> <0BC9D25E-639A-4305-A51A-222AE645152C@lurchi.franken.de> Date: Mon, 16 Dec 2013 19:25:27 -0800 X-Google-Sender-Auth: PRrz0AzVKBqUZz0ezbCYfu1exOk Message-ID: Subject: Re: A small fix for if_em.c, if_igb.c, if_ixgbe.c From: Adrian Chadd To: Michael Tuexen Content-Type: text/plain; charset=ISO-8859-1 Cc: Yong-Hyeon Pyun , FreeBSD Net , John-Mark Gurney , Jack F Vogel , John Baldwin X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2013 03:25:29 -0000 On 16 December 2013 13:04, Michael Tuexen wrote: > On Dec 16, 2013, at 9:15 PM, Adrian Chadd wrote: > >> On 16 December 2013 12:06, Michael Tuexen >> wrote: >> >>>> i agree. if_transmit() should return 0 only if: >>>> >>>> * the driver queued it internally and intends to try transmitting it later; >>>> * the driver directly dispatched the frame to the hardware. >>>> >>>> If it failed to do either of the above, it should return an error. >>>> >>>> How's that sound? >>> That sounds good. However, The transport layer is interested in the case >>> where if_transmit() returns a non-zero value. >>> Does your statement imply: >>> if_transmit() returns a non-zero value only if the packet will not >>> make it on the wire (for example, it failed to queue it). >> >> If there's a queuing layer in the middle then we can't know that for >> certain. If the driver can't transmit the frame (eg it fails because >> of collisions, for example) then again, we can't know that for >> certain. >> >> What we can only know is that it was either queued and may or may not >> make it on the wire, or it wasn't queued/transmitted and it definitely >> _won't_ make it on the wire. > Correct. And I'm only interested in the "it wasn't queued/transmitted > and it definitely _won't_ make it on the wire." part. > So I would need something like > > if_transmit() returns an error only if it wasn't queued/transmitted > and it definitely _won't_ make it on the wire. > > Acceptable for you? Sounds like the same thing to me, so yes. :) -a From owner-freebsd-net@FreeBSD.ORG Tue Dec 17 15:43:50 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5012C9C6; Tue, 17 Dec 2013 15:43:50 +0000 (UTC) Received: from mail-qc0-x234.google.com (mail-qc0-x234.google.com [IPv6:2607:f8b0:400d:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EF8511045; Tue, 17 Dec 2013 15:43:49 +0000 (UTC) Received: by mail-qc0-f180.google.com with SMTP id w7so5083791qcr.11 for ; Tue, 17 Dec 2013 07:43:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=CiPWYMSMTosjx/3PFb5HFsr9t0zfKGDXSvNGufX1n4Q=; b=ppm3gIM3p+jpWYmZ/o74FhJBvJ2fZuO2NQ09yR3imj9DbK9t0TGPvVUfXD9lNBKnOo DPB2Nn3MmiTB5xfFAkSLt17OHuTFMHlF2ScunnBFra8JnmZ/k9nbBANCC7Bg0Olpf+6N Iv3BGfwiBMIpNUd4ClM6cTXOQLZpeCxDFw3EjRhHJIxAUfviNIFdWgy/ul5vjuUsjj3K If6U5aGFoTvoHehJN0S1IkvSuop3Fr0yH7Fp2kx6BqfZ13YAvjF4z1gHlWxxSQ1gRuNm Aru3VEF4acGdTnXDMG5jRZRnDJqZiJuatlZM8y1dXtCt5T2SrhCnwMttnA8vz6GVZSH8 hAdw== MIME-Version: 1.0 X-Received: by 10.224.89.73 with SMTP id d9mr44860263qam.5.1387295029171; Tue, 17 Dec 2013 07:43:49 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Tue, 17 Dec 2013 07:43:49 -0800 (PST) In-Reply-To: References: Date: Tue, 17 Dec 2013 07:43:49 -0800 X-Google-Sender-Auth: 7JlznGHsXQm95Jvv87nfkG50W10 Message-ID: Subject: Re: buf_ring in HEAD is racy From: Adrian Chadd To: Ryan Stone , Jack F Vogel Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2013 15:43:50 -0000 ie: Index: sys/dev/ixgbe/ixgbe.c =================================================================== --- sys/dev/ixgbe/ixgbe.c (revision 2995) +++ sys/dev/ixgbe/ixgbe.c (working copy) @@ -5178,6 +5178,7 @@ struct ixgbe_hw *hw = &adapter->hw; u32 missed_rx = 0, bprc, lxon, lxoff, total; u64 total_missed_rx = 0; + u64 odrops; adapter->stats.crcerrs += IXGBE_READ_REG(hw, IXGBE_CRCERRS); adapter->stats.illerrc += IXGBE_READ_REG(hw, IXGBE_ILLERRC); @@ -5308,6 +5309,11 @@ adapter->stats.fcoedwtc += IXGBE_READ_REG(hw, IXGBE_FCOEDWTC); } + /* TX drops */ + for (int i = 0; i < adapter->num_queues; i++) { + odrops += adapter->tx_rings[i].br->br_drops; + } + /* Fill out the OS statistics structure */ ifp->if_ipackets = adapter->stats.gprc; ifp->if_opackets = adapter->stats.gptc; @@ -5317,6 +5323,9 @@ ifp->if_omcasts = adapter->stats.mptc; ifp->if_collisions = 0; + /* TX drops are stored in if_snd for now, not the top level counters */ + ifp->if_snd.ifq_drops = drops; + /* Rx Errors */ ifp->if_iqdrops = total_missed_rx; ifp->if_ierrors = adapter->stats.crcerrs + adapter->stats.rlec; -adrian On 16 December 2013 12:19, Adrian Chadd wrote: > So I've done some digging. > > The TL;DR - we really should expose the drops count in the ifa_data > field. But that would involve an ABI change and .. well, that will be > full of drama. > > The non-TL:DR: > > * The send queue drops are in the ifmibdata structure, which includes > a few other things. But that requires you use the MIB interface to > pull things out, rather than getifaddrs(). > * getifaddrs() doesn't contain the max sendq length or drops, so we > can't report those without adding a MIB fetch for the relevant > interface. > * bsnmp (which we use at work) correctly populates output discards by > checking the MIB for ifmd_snd_drops. > > So I'd rather we fix ixgbe and other drivers by updating the send > queue drops counter if the frames are dropped. That way it's > consistent with other things. > > We should then do some of these things: > > * add a send queue drops field to the ifdata/ifadata where appropriate > and make sure it gets correctly populated; > * teach netstat to use ifmib instead of getifaddrs(); > * as part of killing the ifsnd queue stuff, we should also migrate > that particular drops counter out from there and to a top-level > counter in ifnet, like the rest. > > Comments? > > > -a > > > On 14 December 2013 16:06, Adrian Chadd wrote: >> oh cool, you just did the output-drops thing I was about to code up. >> We're missing those counters at work and the ops guys poked me about >> it. >> >> I'll also give that a whirl locally and see about working with jack to >> get it into -HEAD / MFC'ed to 10. >> >> Thanks! >> >> >> -a >> >> >> On 13 December 2013 21:04, Ryan Stone wrote: >>> I am seeing spurious output packet drops that appear to be due to >>> insufficient memory barriers in buf_ring. I believe that this is the >>> scenario that I am seeing: >>> >>> 1) The buf_ring is empty, br_prod_head = br_cons_head = 0 >>> 2) Thread 1 attempts to enqueue an mbuf on the buf_ring. It fetches >>> br_prod_head (0) into a local variable called prod_head >>> 3) Thread 2 enqueues an mbuf on the buf_ring. The sequence of events >>> is essentially: >>> >>> Thread 2 claims an index in the ring and atomically sets br_prod_head (say to 1) >>> Thread 2 sets br_ring[1] = mbuf; >>> Thread 2 does a full memory barrier >>> Thread 2 updates br_prod_tail to 1 >>> >>> 4) Thread 2 dequeues the packet from the buf_ring using the >>> single-consumer interface. The sequence of events is essentialy: >>> >>> Thread 2 checks whether queue is empty (br_cons_head == br_prod_tail), >>> this is false >>> Thread 2 sets br_cons_head to 1 >>> Thread 2 grabs the mbuf from br_ring[1] >>> Thread 2 sets br_cons_tail to 1 >>> >>> 5) Thread 1, which is still attempting to enqueue an mbuf on the ring. >>> fetches br_cons_tail (1) into a local variable called cons_tail. It >>> sees cons_tail == 1 but prod_head == 0 and concludes that the ring is >>> full and drops the packet (incrementing br_drops unatomically, I might >>> add) >>> >>> >>> I can reproduce several drops per minute by configuring the ixgbe >>> driver to use only 1 queue and then sending traffic from concurrent 8 >>> iperf processes. (You will need this hacky patch to even see the >>> drops with netstat, though: >>> http://people.freebsd.org/~rstone/patches/ixgbe_br_drops.diff) >>> >>> I am investigating fixing buf_ring by using acquire/release semantics >>> rather than load/store barriers. However, I note that this will >>> apparently be the second attempt to fix buf_ring, and I'm seriously >>> questioning whether this is worth the effort compared to the >>> simplicity of using a mutex. I'm not even convinced that a correct >>> lockless implementation will even be a performance win, given the >>> number of memory barriers that will apparently be necessary. >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Dec 17 16:29:12 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D78BD481 for ; Tue, 17 Dec 2013 16:29:12 +0000 (UTC) Received: from mail01.cae.com (mail01.cae.com [142.39.204.151]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A3DAD1435 for ; Tue, 17 Dec 2013 16:29:12 +0000 (UTC) Received: from caehub03.caecorp.cae.com (caehub03.caecorp.cae.com [142.39.249.107]) by mail01.cae.com (8.14.4/8.14.4) with ESMTP id rBHGRSTt014750 for ; Tue, 17 Dec 2013 11:27:28 -0500 Received: from caehub01.caecorp.cae.com (142.39.21.50) by caehub03.caecorp.cae.com (142.39.249.107) with Microsoft SMTP Server (TLS) id 8.3.342.0; Tue, 17 Dec 2013 11:27:28 -0500 Received: from CAEMEX81.caecorp.cae.com ([142.39.19.46]) by caehub01.caecorp.cae.com ([142.39.21.50]) with mapi; Tue, 17 Dec 2013 11:27:28 -0500 From: Claude Marinier To: "freebsd-net@freebsd.org" Date: Tue, 17 Dec 2013 11:27:38 -0500 Subject: em3 no carrier Thread-Topic: em3 no carrier Thread-Index: Ac77RLxbzAmisN0hSYezwlxENoWNSA== Message-ID: <34E38D657499684D9EBAA6D73CD2D1E21958EE48F8@CAEMEX81.caecorp.cae.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US X-TM-AS-Product-Ver: SMEX-10.2.0.3176-7.000.1014-20368.000 X-TM-AS-Result: No--5.607900-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2013 16:29:12 -0000 Hi, This is a WAN emulator (ipfw, DummyNet, and BIRD). FreeBSD 9.2 Release (amd= 64) is running on DL360 G5 with a new NC364T quad-port Ethernet NIC. I inst= alled the new NIC yesterday. The four ports are connected to routers in a l= ab. FreeBSD WAMemu 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898: Thu Sep 26 22:50= :31 UTC 2013 root@bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 Dec 16 17:13:19 WANemu kernel: em3: port 0x6020-0x603f mem 0xfdee0000-0xfdefffff,0xfde00000-0xfde7ffff ir= q 17 at device 0.1 on pci22 Dec 16 17:13:19 WANemu kernel: em3: Using an MSI interrupt Dec 16 17:13:19 WANemu kernel: em3: Ethernet address: e8:39:35:13:21:6e The four em interfaces are configured the same way (lines in rc.conf are id= entical except for IP address. ifconfig_em3=3D"inet x.y.113.197/29 media 100baseTX mediaopt full-duplex" ifconfig_em0=3D"inet x.y.113.14/29 media 100baseTX mediaopt full-duplex" ifconfig_em2=3D"inet x.y.113.109/29 media 100baseTX mediaopt full-duplex" ifconfig_em1=3D"inet x.y.113.189/29 media 100baseTX mediaopt full-duplex" The GE 0/1 port on the Cisco 2821 shows no link and ifconfig shows "no carr= ier" for em3. Oddly, it also shows autoselect. em0: flags=3D8843 metric 0 mtu 1500 options=3D4019b ether e8:39:35:13:21:6d inet x.y.113.14 netmask 0xfffffff8 broadcast x.y.113.15 inet6 fe80::ea39:35ff:fe13:216d%em0 prefixlen 64 scopeid 0x1 nd6 options=3D29 media: Ethernet 100baseTX status: active em1: flags=3D8843 metric 0 mtu 1500 options=3D4019b ether e8:39:35:13:21:6c inet x.y.113.189 netmask 0xfffffff8 broadcast x.y.113.191 inet6 fe80::ea39:35ff:fe13:216c%em1 prefixlen 64 scopeid 0x2 nd6 options=3D29 media: Ethernet 100baseTX status: active em2: flags=3D8843 metric 0 mtu 1500 options=3D4019b ether e8:39:35:13:21:6f inet 131.140.113.109 netmask 0xfffffff8 broadcast 131.140.113.111 inet6 fe80::ea39:35ff:fe13:216f%em2 prefixlen 64 scopeid 0x3 nd6 options=3D29 media: Ethernet 100baseTX status: active em3: flags=3D8843 metric 0 mtu 1500 options=3D4019b ether e8:39:35:13:21:6e inet 131.140.113.197 netmask 0xfffffff8 broadcast 131.140.113.199 inet6 fe80::ea39:35ff:fe13:216e%em3 prefixlen 64 scopeid 0x4 nd6 options=3D29 media: Ethernet 100baseTX (autoselect) status: no carrier I have tried three Ethernet cables with no difference. This morning, I brie= fly booted Puppy Linux from USB and manually configured the four Ethernet i= nterfaces. They all work (ping good to all four routers); the Cisco on em3 = shows happy lights. I then booted back into FreeBSD, the problem with em3 r= emains. I have not had any success searching with Google; I may not have us= ed suitable search terms. One more odd symptom: bmon has trouble with these interfaces. em0=E895^S!m on WANemu Name qqqqqqqqqqqqqqqqqqqqqqqq WANemu (local) 0 em0=E895^S!m 1 em1=E895^S!l 2 em2=E895^S!o 3 em3=E895^S!n 4 usbus0 5 usbus1 6 usbus2 7 usbus3 8 usbus4 9 usbus5 10 ipfw0 11 lo0 It looks like FreeBSD is somehow mishandling these interfaces. Before purch= asing it, I searched with Google and found reports of people using this dev= ice without problems. I do not know how to proceed. Thank you for your help. -- C Marinier From owner-freebsd-net@FreeBSD.ORG Tue Dec 17 16:39:50 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9322B6AF; Tue, 17 Dec 2013 16:39:50 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A11FE152A; Tue, 17 Dec 2013 16:39:49 +0000 (UTC) Received: from [192.168.1.200] (p508F1D30.dip0.t-ipconnect.de [80.143.29.48]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id BDAB31C0C0692; Tue, 17 Dec 2013 17:39:45 +0100 (CET) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) Subject: Re: buf_ring in HEAD is racy From: Michael Tuexen In-Reply-To: Date: Tue, 17 Dec 2013 17:39:45 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Adrian Chadd X-Mailer: Apple Mail (2.1510) Cc: Jack F Vogel , freebsd-net , Ryan Stone X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2013 16:39:50 -0000 On Dec 17, 2013, at 4:43 PM, Adrian Chadd wrote: > ie: >=20 > Index: sys/dev/ixgbe/ixgbe.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/dev/ixgbe/ixgbe.c (revision 2995) > +++ sys/dev/ixgbe/ixgbe.c (working copy) > @@ -5178,6 +5178,7 @@ > struct ixgbe_hw *hw =3D &adapter->hw; > u32 missed_rx =3D 0, bprc, lxon, lxoff, total; > u64 total_missed_rx =3D 0; > + u64 odrops; What about=20 + u64 odrops =3D 0; since odrops seems to be used uninitialized. Best regards Michael >=20 > adapter->stats.crcerrs +=3D IXGBE_READ_REG(hw, IXGBE_CRCERRS); > adapter->stats.illerrc +=3D IXGBE_READ_REG(hw, IXGBE_ILLERRC); > @@ -5308,6 +5309,11 @@ > adapter->stats.fcoedwtc +=3D IXGBE_READ_REG(hw, = IXGBE_FCOEDWTC); > } >=20 > + /* TX drops */ > + for (int i =3D 0; i < adapter->num_queues; i++) { > + odrops +=3D adapter->tx_rings[i].br->br_drops; > + } > + > /* Fill out the OS statistics structure */ > ifp->if_ipackets =3D adapter->stats.gprc; > ifp->if_opackets =3D adapter->stats.gptc; > @@ -5317,6 +5323,9 @@ > ifp->if_omcasts =3D adapter->stats.mptc; > ifp->if_collisions =3D 0; >=20 > + /* TX drops are stored in if_snd for now, not the top level = counters */ > + ifp->if_snd.ifq_drops =3D drops; > + > /* Rx Errors */ > ifp->if_iqdrops =3D total_missed_rx; > ifp->if_ierrors =3D adapter->stats.crcerrs + = adapter->stats.rlec; >=20 >=20 >=20 >=20 > -adrian >=20 > On 16 December 2013 12:19, Adrian Chadd wrote: >> So I've done some digging. >>=20 >> The TL;DR - we really should expose the drops count in the ifa_data >> field. But that would involve an ABI change and .. well, that will be >> full of drama. >>=20 >> The non-TL:DR: >>=20 >> * The send queue drops are in the ifmibdata structure, which includes >> a few other things. But that requires you use the MIB interface to >> pull things out, rather than getifaddrs(). >> * getifaddrs() doesn't contain the max sendq length or drops, so we >> can't report those without adding a MIB fetch for the relevant >> interface. >> * bsnmp (which we use at work) correctly populates output discards by >> checking the MIB for ifmd_snd_drops. >>=20 >> So I'd rather we fix ixgbe and other drivers by updating the send >> queue drops counter if the frames are dropped. That way it's >> consistent with other things. >>=20 >> We should then do some of these things: >>=20 >> * add a send queue drops field to the ifdata/ifadata where = appropriate >> and make sure it gets correctly populated; >> * teach netstat to use ifmib instead of getifaddrs(); >> * as part of killing the ifsnd queue stuff, we should also migrate >> that particular drops counter out from there and to a top-level >> counter in ifnet, like the rest. >>=20 >> Comments? >>=20 >>=20 >> -a >>=20 >>=20 >> On 14 December 2013 16:06, Adrian Chadd wrote: >>> oh cool, you just did the output-drops thing I was about to code up. >>> We're missing those counters at work and the ops guys poked me about >>> it. >>>=20 >>> I'll also give that a whirl locally and see about working with jack = to >>> get it into -HEAD / MFC'ed to 10. >>>=20 >>> Thanks! >>>=20 >>>=20 >>> -a >>>=20 >>>=20 >>> On 13 December 2013 21:04, Ryan Stone wrote: >>>> I am seeing spurious output packet drops that appear to be due to >>>> insufficient memory barriers in buf_ring. I believe that this is = the >>>> scenario that I am seeing: >>>>=20 >>>> 1) The buf_ring is empty, br_prod_head =3D br_cons_head =3D 0 >>>> 2) Thread 1 attempts to enqueue an mbuf on the buf_ring. It = fetches >>>> br_prod_head (0) into a local variable called prod_head >>>> 3) Thread 2 enqueues an mbuf on the buf_ring. The sequence of = events >>>> is essentially: >>>>=20 >>>> Thread 2 claims an index in the ring and atomically sets = br_prod_head (say to 1) >>>> Thread 2 sets br_ring[1] =3D mbuf; >>>> Thread 2 does a full memory barrier >>>> Thread 2 updates br_prod_tail to 1 >>>>=20 >>>> 4) Thread 2 dequeues the packet from the buf_ring using the >>>> single-consumer interface. The sequence of events is essentialy: >>>>=20 >>>> Thread 2 checks whether queue is empty (br_cons_head =3D=3D = br_prod_tail), >>>> this is false >>>> Thread 2 sets br_cons_head to 1 >>>> Thread 2 grabs the mbuf from br_ring[1] >>>> Thread 2 sets br_cons_tail to 1 >>>>=20 >>>> 5) Thread 1, which is still attempting to enqueue an mbuf on the = ring. >>>> fetches br_cons_tail (1) into a local variable called cons_tail. = It >>>> sees cons_tail =3D=3D 1 but prod_head =3D=3D 0 and concludes that = the ring is >>>> full and drops the packet (incrementing br_drops unatomically, I = might >>>> add) >>>>=20 >>>>=20 >>>> I can reproduce several drops per minute by configuring the ixgbe >>>> driver to use only 1 queue and then sending traffic from concurrent = 8 >>>> iperf processes. (You will need this hacky patch to even see the >>>> drops with netstat, though: >>>> http://people.freebsd.org/~rstone/patches/ixgbe_br_drops.diff) >>>>=20 >>>> I am investigating fixing buf_ring by using acquire/release = semantics >>>> rather than load/store barriers. However, I note that this will >>>> apparently be the second attempt to fix buf_ring, and I'm seriously >>>> questioning whether this is worth the effort compared to the >>>> simplicity of using a mutex. I'm not even convinced that a correct >>>> lockless implementation will even be a performance win, given the >>>> number of memory barriers that will apparently be necessary. >>>> _______________________________________________ >>>> freebsd-net@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net >>>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Tue Dec 17 16:42:17 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B8E288F5 for ; Tue, 17 Dec 2013 16:42:17 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4E10C15A4 for ; Tue, 17 Dec 2013 16:42:17 +0000 (UTC) Received: from [192.168.1.200] (p508F1D30.dip0.t-ipconnect.de [80.143.29.48]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 95C521C0C0692; Tue, 17 Dec 2013 17:42:15 +0100 (CET) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) Subject: Re: em3 no carrier From: Michael Tuexen In-Reply-To: <34E38D657499684D9EBAA6D73CD2D1E21958EE48F8@CAEMEX81.caecorp.cae.com> Date: Tue, 17 Dec 2013 17:42:15 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <34E38D657499684D9EBAA6D73CD2D1E21958EE48F8@CAEMEX81.caecorp.cae.com> To: Claude Marinier X-Mailer: Apple Mail (2.1510) Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2013 16:42:17 -0000 On Dec 17, 2013, at 5:27 PM, Claude Marinier = wrote: > Hi, >=20 > This is a WAN emulator (ipfw, DummyNet, and BIRD). FreeBSD 9.2 Release = (amd64) is running on DL360 G5 with a new NC364T quad-port Ethernet NIC. = I installed the new NIC yesterday. The four ports are connected to = routers in a lab. > FreeBSD WAMemu 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898: Thu Sep 26 = 22:50:31 UTC 2013 > root@bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 >=20 > Dec 16 17:13:19 WANemu kernel: em3: port 0x6020-0x603f mem = 0xfdee0000-0xfdefffff,0xfde00000-0xfde7ffff irq 17 at device 0.1 on = pci22 > Dec 16 17:13:19 WANemu kernel: em3: Using an MSI interrupt > Dec 16 17:13:19 WANemu kernel: em3: Ethernet address: = e8:39:35:13:21:6e >=20 > The four em interfaces are configured the same way (lines in rc.conf = are identical except for IP address. > ifconfig_em3=3D"inet x.y.113.197/29 media 100baseTX mediaopt = full-duplex" > ifconfig_em0=3D"inet x.y.113.14/29 media 100baseTX mediaopt = full-duplex" > ifconfig_em2=3D"inet x.y.113.109/29 media 100baseTX mediaopt = full-duplex" > ifconfig_em1=3D"inet x.y.113.189/29 media 100baseTX mediaopt = full-duplex" >=20 > The GE 0/1 port on the Cisco 2821 shows no link and ifconfig shows "no = carrier" for em3. Oddly, it also shows autoselect. So configure it on the Cisco box also as 100MBit, full duplex. Does it = work? Best regards Michael > em0: flags=3D8843 metric 0 mtu = 1500 > = options=3D4019b > ether e8:39:35:13:21:6d > inet x.y.113.14 netmask 0xfffffff8 broadcast x.y.113.15 > inet6 fe80::ea39:35ff:fe13:216d%em0 prefixlen 64 scopeid 0x1 > nd6 options=3D29 > media: Ethernet 100baseTX > status: active > em1: flags=3D8843 metric 0 mtu = 1500 > = options=3D4019b > ether e8:39:35:13:21:6c > inet x.y.113.189 netmask 0xfffffff8 broadcast x.y.113.191 > inet6 fe80::ea39:35ff:fe13:216c%em1 prefixlen 64 scopeid 0x2 > nd6 options=3D29 > media: Ethernet 100baseTX > status: active > em2: flags=3D8843 metric 0 mtu = 1500 > = options=3D4019b > ether e8:39:35:13:21:6f > inet 131.140.113.109 netmask 0xfffffff8 broadcast 131.140.113.111 > inet6 fe80::ea39:35ff:fe13:216f%em2 prefixlen 64 scopeid 0x3 > nd6 options=3D29 > media: Ethernet 100baseTX > status: active > em3: flags=3D8843 metric 0 mtu = 1500 > = options=3D4019b > ether e8:39:35:13:21:6e > inet 131.140.113.197 netmask 0xfffffff8 broadcast 131.140.113.199 > inet6 fe80::ea39:35ff:fe13:216e%em3 prefixlen 64 scopeid 0x4 > nd6 options=3D29 > media: Ethernet 100baseTX (autoselect) > status: no carrier >=20 > I have tried three Ethernet cables with no difference. This morning, I = briefly booted Puppy Linux from USB and manually configured the four = Ethernet interfaces. They all work (ping good to all four routers); the = Cisco on em3 shows happy lights. I then booted back into FreeBSD, the = problem with em3 remains. I have not had any success searching with = Google; I may not have used suitable search terms. >=20 > One more odd symptom: bmon has trouble with these interfaces. > em0=E895^S!m on WANemu > Name > qqqqqqqqqqqqqqqqqqqqqqqq > WANemu (local) > 0 em0=E895^S!m > 1 em1=E895^S!l > 2 em2=E895^S!o > 3 em3=E895^S!n > 4 usbus0 > 5 usbus1 > 6 usbus2 > 7 usbus3 > 8 usbus4 > 9 usbus5 > 10 ipfw0 > 11 lo0 >=20 > It looks like FreeBSD is somehow mishandling these interfaces. Before = purchasing it, I searched with Google and found reports of people using = this device without problems. I do not know how to proceed. >=20 > Thank you for your help. >=20 > -- > C Marinier >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Tue Dec 17 16:44:59 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 80024CA3; Tue, 17 Dec 2013 16:44:59 +0000 (UTC) Received: from mail-qc0-x234.google.com (mail-qc0-x234.google.com [IPv6:2607:f8b0:400d:c01::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2904215E0; Tue, 17 Dec 2013 16:44:59 +0000 (UTC) Received: by mail-qc0-f180.google.com with SMTP id w7so5100531qcr.39 for ; Tue, 17 Dec 2013 08:44:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=5+pCxtB6tBJ9pluRGGTxQ62hyhBZLasdw63NWDXXP00=; b=VapWYd8OUq3Z7xdfc7NNDTA91gNx5nVnQCk6ffkwEok/Mhac1EqonmzqmZ+8O1ke/4 Nb0aQong6/l3s2jNktgwa6qKFlfPziXIZg9ur4yuu74MxEMPF8tSKj1ZGPpwpxMyV278 XuYecd67qb9f0qzvzTpE+5fIN7M756R+wcSRxhIb6OQCJTSMfen5QsdQpQNQdijabM+1 YPuvspWhJObTO56XmVil2/+XYSEQV2wpWbJQDcAUu82VZJCpYxZcb3cjlcZbPwHjnuZn Tb8plaau2yFLbIsngY3JFA+Gwo0xWxM7utmNFloIgpReFj+qUpVPFlDWzWpkZXE9XgDf qnDg== MIME-Version: 1.0 X-Received: by 10.224.67.200 with SMTP id s8mr44412259qai.75.1387298698354; Tue, 17 Dec 2013 08:44:58 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Tue, 17 Dec 2013 08:44:58 -0800 (PST) In-Reply-To: References: Date: Tue, 17 Dec 2013 08:44:58 -0800 X-Google-Sender-Auth: g-koZA_OfwUCXA7Yw_lAF1HVN4k Message-ID: Subject: Re: buf_ring in HEAD is racy From: Adrian Chadd To: Michael Tuexen Content-Type: text/plain; charset=ISO-8859-1 Cc: Jack F Vogel , freebsd-net , Ryan Stone X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2013 16:44:59 -0000 Eep, missed that. I'll go and fix that now. -a From owner-freebsd-net@FreeBSD.ORG Tue Dec 17 16:46:45 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B87B0E9B for ; Tue, 17 Dec 2013 16:46:45 +0000 (UTC) Received: from smtp6-g21.free.fr (smtp6-g21.free.fr [IPv6:2a01:e0c:1:1599::15]) by mx1.freebsd.org (Postfix) with ESMTP id 5805E1605 for ; Tue, 17 Dec 2013 16:46:43 +0000 (UTC) Received: from oldfaithful.bebik.local (unknown [82.227.164.69]) by smtp6-g21.free.fr (Postfix) with ESMTP id 9BE5082305; Tue, 17 Dec 2013 17:46:36 +0100 (CET) Received: by oldfaithful.bebik.local (Postfix, from userid 1001) id 94CC9915103; Tue, 17 Dec 2013 17:39:36 +0100 (CET) Date: Tue, 17 Dec 2013 17:39:36 +0100 From: Rodrigo Osorio To: Claude Marinier Subject: Re: em3 no carrier Message-ID: <20131217163936.GA35856@oldfaithful.bebik.local> References: <34E38D657499684D9EBAA6D73CD2D1E21958EE48F8@CAEMEX81.caecorp.cae.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <34E38D657499684D9EBAA6D73CD2D1E21958EE48F8@CAEMEX81.caecorp.cae.com> User-Agent: Mutt/1.4.2.3i Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2013 16:46:45 -0000 Hi, Can you give us more informations about the network card / chips ? Did you found any error in /var/log/ ? - rodrigo On 17/12/13 11:27 -0500, Claude Marinier wrote: > Hi, > > This is a WAN emulator (ipfw, DummyNet, and BIRD). FreeBSD 9.2 Release (amd64) is running on DL360 G5 with a new NC364T quad-port Ethernet NIC. I installed the new NIC yesterday. The four ports are connected to routers in a lab. > FreeBSD WAMemu 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898: Thu Sep 26 22:50:31 UTC 2013 > root@bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > > Dec 16 17:13:19 WANemu kernel: em3: port 0x6020-0x603f mem 0xfdee0000-0xfdefffff,0xfde00000-0xfde7ffff irq 17 at device 0.1 on pci22 > Dec 16 17:13:19 WANemu kernel: em3: Using an MSI interrupt > Dec 16 17:13:19 WANemu kernel: em3: Ethernet address: e8:39:35:13:21:6e > > The four em interfaces are configured the same way (lines in rc.conf are identical except for IP address. > ifconfig_em3="inet x.y.113.197/29 media 100baseTX mediaopt full-duplex" > ifconfig_em0="inet x.y.113.14/29 media 100baseTX mediaopt full-duplex" > ifconfig_em2="inet x.y.113.109/29 media 100baseTX mediaopt full-duplex" > ifconfig_em1="inet x.y.113.189/29 media 100baseTX mediaopt full-duplex" > > The GE 0/1 port on the Cisco 2821 shows no link and ifconfig shows "no carrier" for em3. Oddly, it also shows autoselect. > em0: flags=8843 metric 0 mtu 1500 > options=4019b > ether e8:39:35:13:21:6d > inet x.y.113.14 netmask 0xfffffff8 broadcast x.y.113.15 > inet6 fe80::ea39:35ff:fe13:216d%em0 prefixlen 64 scopeid 0x1 > nd6 options=29 > media: Ethernet 100baseTX > status: active > em1: flags=8843 metric 0 mtu 1500 > options=4019b > ether e8:39:35:13:21:6c > inet x.y.113.189 netmask 0xfffffff8 broadcast x.y.113.191 > inet6 fe80::ea39:35ff:fe13:216c%em1 prefixlen 64 scopeid 0x2 > nd6 options=29 > media: Ethernet 100baseTX > status: active > em2: flags=8843 metric 0 mtu 1500 > options=4019b > ether e8:39:35:13:21:6f > inet 131.140.113.109 netmask 0xfffffff8 broadcast 131.140.113.111 > inet6 fe80::ea39:35ff:fe13:216f%em2 prefixlen 64 scopeid 0x3 > nd6 options=29 > media: Ethernet 100baseTX > status: active > em3: flags=8843 metric 0 mtu 1500 > options=4019b > ether e8:39:35:13:21:6e > inet 131.140.113.197 netmask 0xfffffff8 broadcast 131.140.113.199 > inet6 fe80::ea39:35ff:fe13:216e%em3 prefixlen 64 scopeid 0x4 > nd6 options=29 > media: Ethernet 100baseTX (autoselect) > status: no carrier > > I have tried three Ethernet cables with no difference. This morning, I briefly booted Puppy Linux from USB and manually configured the four Ethernet interfaces. They all work (ping good to all four routers); the Cisco on em3 shows happy lights. I then booted back into FreeBSD, the problem with em3 remains. I have not had any success searching with Google; I may not have used suitable search terms. > > One more odd symptom: bmon has trouble with these interfaces. > em0č95^S!m on WANemu > Name > qqqqqqqqqqqqqqqqqqqqqqqq > WANemu (local) > 0 em0č95^S!m > 1 em1č95^S!l > 2 em2č95^S!o > 3 em3č95^S!n > 4 usbus0 > 5 usbus1 > 6 usbus2 > 7 usbus3 > 8 usbus4 > 9 usbus5 > 10 ipfw0 > 11 lo0 > > It looks like FreeBSD is somehow mishandling these interfaces. Before purchasing it, I searched with Google and found reports of people using this device without problems. I do not know how to proceed. > > Thank you for your help. > > -- > C Marinier > > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Dec 17 16:49:51 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 523C3AB; Tue, 17 Dec 2013 16:49:51 +0000 (UTC) Received: from mail-qc0-x236.google.com (mail-qc0-x236.google.com [IPv6:2607:f8b0:400d:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EF8401647; Tue, 17 Dec 2013 16:49:50 +0000 (UTC) Received: by mail-qc0-f182.google.com with SMTP id e16so5188566qcx.27 for ; Tue, 17 Dec 2013 08:49:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=CRNLvC4cf31he471kdl8nWCivf0kyKV1StiPcgf9qFc=; b=HYG9LT6cC4+yQz4ue5UytRHP93POzpbX9js6UXL1uyGwAX5+IAOvsHS9Ku4Z2nL2ey 0DA1RwoWYYBHLphxjSKorHF8Vjzk2bAZB8OZqfmcqadL0Eba06ufiJldSNudJfq691ld xRsdnDSFPOt/5tFc0q9hBQ67VFVT2+UpTYSqjNaA5d6IEVZngGjx/m7T4HftKoDSitW9 exeVQbGOMujwHiqXa9bMUU/U5EFYZBgMQ6aD6wDUafv+MQVq3UlOdtfq9Eyk3o8q/7yC fBTM0Y61ikp1Je2Xqd/p8htuIIrUVOQRz1zUOgxGkAhDgA68V8NSW+Hrk8QS7LOzMK09 enLw== MIME-Version: 1.0 X-Received: by 10.224.5.69 with SMTP id 5mr44844723qau.95.1387298990020; Tue, 17 Dec 2013 08:49:50 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Tue, 17 Dec 2013 08:49:49 -0800 (PST) In-Reply-To: References: Date: Tue, 17 Dec 2013 08:49:49 -0800 X-Google-Sender-Auth: UBJ9vBRx0bCwkOfGN5nspC41RrM Message-ID: Subject: Re: buf_ring in HEAD is racy From: Adrian Chadd To: Michael Tuexen Content-Type: text/plain; charset=ISO-8859-1 Cc: Jack F Vogel , freebsd-net , Ryan Stone X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2013 16:49:51 -0000 Try: Index: sys/dev/ixgbe/ixgbe.c =================================================================== --- sys/dev/ixgbe/ixgbe.c (revision 2995) +++ sys/dev/ixgbe/ixgbe.c (working copy) @@ -5178,6 +5178,7 @@ struct ixgbe_hw *hw = &adapter->hw; u32 missed_rx = 0, bprc, lxon, lxoff, total; u64 total_missed_rx = 0; + u64 odrops = 0; adapter->stats.crcerrs += IXGBE_READ_REG(hw, IXGBE_CRCERRS); adapter->stats.illerrc += IXGBE_READ_REG(hw, IXGBE_ILLERRC); @@ -5308,6 +5309,11 @@ adapter->stats.fcoedwtc += IXGBE_READ_REG(hw, IXGBE_FCOEDWTC); } + /* TX drops */ + for (int i = 0; i < adapter->num_queues; i++) { + odrops += adapter->tx_rings[i].br->br_drops; + } + /* Fill out the OS statistics structure */ ifp->if_ipackets = adapter->stats.gprc; ifp->if_opackets = adapter->stats.gptc; @@ -5317,6 +5323,9 @@ ifp->if_omcasts = adapter->stats.mptc; ifp->if_collisions = 0; + /* TX drops are stored in if_snd for now, not the top level counters */ + ifp->if_snd.ifq_drops = odrops; + /* Rx Errors */ ifp->if_iqdrops = total_missed_rx; ifp->if_ierrors = adapter->stats.crcerrs + adapter->stats.rlec; -adrian From owner-freebsd-net@FreeBSD.ORG Tue Dec 17 17:14:56 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 47C45811 for ; Tue, 17 Dec 2013 17:14:56 +0000 (UTC) Received: from mail02.cae.com (mail02.cae.com [142.39.204.152]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 169C7183F for ; Tue, 17 Dec 2013 17:14:55 +0000 (UTC) Received: from caehub03.caecorp.cae.com (caehub04.caecorp.cae.com [142.39.249.106]) by mail02.cae.com (8.14.4/8.14.4) with ESMTP id rBHHCnqM002411; Tue, 17 Dec 2013 12:12:49 -0500 Received: from CAEHUB02.caecorp.cae.com (142.39.20.44) by caehub04.caecorp.cae.com (142.39.249.106) with Microsoft SMTP Server (TLS) id 8.3.342.0; Tue, 17 Dec 2013 12:12:49 -0500 Received: from CAEMEX81.caecorp.cae.com ([142.39.19.46]) by caehub02.caecorp.cae.com ([142.39.20.44]) with mapi; Tue, 17 Dec 2013 12:12:48 -0500 From: Claude Marinier To: Rodrigo Osorio Date: Tue, 17 Dec 2013 12:13:00 -0500 Subject: RE: em3 no carrier Thread-Topic: em3 no carrier Thread-Index: Ac77R5xgd4UTvHIrS92qLvLsTh9mwgAAsJ+w Message-ID: <34E38D657499684D9EBAA6D73CD2D1E21958EE4942@CAEMEX81.caecorp.cae.com> References: <34E38D657499684D9EBAA6D73CD2D1E21958EE48F8@CAEMEX81.caecorp.cae.com> <20131217163936.GA35856@oldfaithful.bebik.local> In-Reply-To: <20131217163936.GA35856@oldfaithful.bebik.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US X-TM-AS-Product-Ver: SMEX-10.2.0.3176-7.000.1014-20368.000 X-TM-AS-Result: No--20.724300-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2013 17:14:56 -0000 > -----Original Message----- > From: Rodrigo Osorio [mailto:rodrigo@bebik.net] > Sent: Tuesday, December 17, 2013 11:40 AM > To: Claude Marinier > Cc: freebsd-net@freebsd.org > Subject: Re: em3 no carrier >=20 > Hi, >=20 > Can you give us more informations about the network card / chips ? > Did you found any error in /var/log/ ? >=20 > - rodrigo My apologies, I missed that bit of information. Yes, there is an error in /var/log/messages. Dec 17 11:33:07 WANemu bird: OSPF: Socket error on em3: No buffer space ava= ilable Dec 17 11:33:47 WANemu last message repeated 4 times Dec 17 11:35:57 WANemu last message repeated 13 times Dec 17 11:45:57 WANemu last message repeated 60 times Dec 17 11:55:47 WANemu last message repeated 59 times Also note that the HP NC364T uses an Intel 82571EB chipset. > On 17/12/13 11:27 -0500, Claude Marinier wrote: > > Hi, > > > > This is a WAN emulator (ipfw, DummyNet, and BIRD). FreeBSD 9.2 Release > (amd64) is running on DL360 G5 with a new NC364T quad-port Ethernet NIC. = I > installed the new NIC yesterday. The four ports are connected to routers = in > a lab. > > FreeBSD WAMemu 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898: Thu Sep > 26 > > 22:50:31 UTC 2013 > > root@bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > > > > Dec 16 17:13:19 WANemu kernel: em3: > Connection 7.3.8> port 0x6020-0x603f mem > > 0xfdee0000-0xfdefffff,0xfde00000-0xfde7ffff irq 17 at device 0.1 on > > pci22 Dec 16 17:13:19 WANemu kernel: em3: Using an MSI interrupt Dec > > 16 17:13:19 WANemu kernel: em3: Ethernet address: e8:39:35:13:21:6e > > > > The four em interfaces are configured the same way (lines in rc.conf ar= e > identical except for IP address. > > ifconfig_em3=3D"inet x.y.113.197/29 media 100baseTX mediaopt full- > duplex" > > ifconfig_em0=3D"inet x.y.113.14/29 media 100baseTX mediaopt full-duplex= " > > ifconfig_em2=3D"inet x.y.113.109/29 media 100baseTX mediaopt full- > duplex" > > ifconfig_em1=3D"inet x.y.113.189/29 media 100baseTX mediaopt full- > duplex" > > > > The GE 0/1 port on the Cisco 2821 shows no link and ifconfig shows "no > carrier" for em3. Oddly, it also shows autoselect. > > em0: flags=3D8843 metric 0 > mtu 1500 > > > options=3D4019b CSUM,TSO4,VLAN_HWTSO> > > ether e8:39:35:13:21:6d > > inet x.y.113.14 netmask 0xfffffff8 broadcast x.y.113.15 > > inet6 fe80::ea39:35ff:fe13:216d%em0 prefixlen 64 scopeid 0x1 > > nd6 options=3D29 > > media: Ethernet 100baseTX > > status: active > > em1: flags=3D8843 metric 0 > mtu 1500 > > > options=3D4019b CSUM,TSO4,VLAN_HWTSO> > > ether e8:39:35:13:21:6c > > inet x.y.113.189 netmask 0xfffffff8 broadcast x.y.113.191 > > inet6 fe80::ea39:35ff:fe13:216c%em1 prefixlen 64 scopeid 0x2 > > nd6 options=3D29 > > media: Ethernet 100baseTX > > status: active > > em2: flags=3D8843 metric 0 > mtu 1500 > > > options=3D4019b CSUM,TSO4,VLAN_HWTSO> > > ether e8:39:35:13:21:6f > > inet 131.140.113.109 netmask 0xfffffff8 broadcast 131.140.113.111 > > inet6 fe80::ea39:35ff:fe13:216f%em2 prefixlen 64 scopeid 0x3 > > nd6 options=3D29 > > media: Ethernet 100baseTX > > status: active > > em3: flags=3D8843 metric 0 > mtu 1500 > > > options=3D4019b CSUM,TSO4,VLAN_HWTSO> > > ether e8:39:35:13:21:6e > > inet 131.140.113.197 netmask 0xfffffff8 broadcast 131.140.113.199 > > inet6 fe80::ea39:35ff:fe13:216e%em3 prefixlen 64 scopeid 0x4 > > nd6 options=3D29 > > media: Ethernet 100baseTX (autoselect) > > status: no carrier > > > > I have tried three Ethernet cables with no difference. This morning, I > briefly booted Puppy Linux from USB and manually configured the four > Ethernet interfaces. They all work (ping good to all four routers); the C= isco > on em3 shows happy lights. I then booted back into FreeBSD, the problem > with em3 remains. I have not had any success searching with Google; I may > not have used suitable search terms. > > > > One more odd symptom: bmon has trouble with these interfaces. > > em0=E895^S!m on WANemu > > Name > > qqqqqqqqqqqqqqqqqqqqqqqq > > WANemu (local) > > 0 em0=E895^S!m > > 1 em1=E895^S!l > > 2 em2=E895^S!o > > 3 em3=E895^S!n > > 4 usbus0 > > 5 usbus1 > > 6 usbus2 > > 7 usbus3 > > 8 usbus4 > > 9 usbus5 > > 10 ipfw0 > > 11 lo0 > > > > It looks like FreeBSD is somehow mishandling these interfaces. Before > purchasing it, I searched with Google and found reports of people using t= his > device without problems. I do not know how to proceed. > > > > Thank you for your help. > > > > -- > > C Marinier > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Dec 17 19:13:29 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BA3DAA80; Tue, 17 Dec 2013 19:13:29 +0000 (UTC) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) by mx1.freebsd.org (Postfix) with ESMTP id 456091289; Tue, 17 Dec 2013 19:13:29 +0000 (UTC) Received: from amavis-proxy-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3dkTXR0dPyzGN2x; Tue, 17 Dec 2013 20:13:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= message-id:content-transfer-encoding:content-type:content-type :mime-version:user-agent:date:date:subject:subject:organization :from:from:received:received:received; s=jakla2; t=1387307601; x=1389899602; bh=SrdiS+397kwoYVVx5GEbJtVtCK1XAJRUEEACx7syEAo=; b= SL1VWv0OUMX8xjwX5PM46s6GqnWoieUVrI6fHmqn3p9KS6fDDIr1D4JQbnzaHasN Rg8Bf9OquYolbeY77wz+JYZSBjWHLrt2UW6fatjGnrf5mI0/vLe6AN7lnnA9UDeh VJShWeikk9iVLyy41UEvaTZbQ+xxrhh1Ye6dyo5tn1Y= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-proxy-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10012) with ESMTP id DjH_k13aax62; Tue, 17 Dec 2013 20:13:21 +0100 (CET) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP; Tue, 17 Dec 2013 20:13:21 +0100 (CET) Received: from neli.ijs.si (neli.ijs.si [IPv6:2001:1470:ff80:88:21c:c0ff:feb1:8c91]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mildred.ijs.si (Postfix) with ESMTPSA id 7AE174F8; Tue, 17 Dec 2013 20:13:21 +0100 (CET) From: Mark Martinec Organization: J. Stefan Institute To: freebsd-current@freebsd.org, freebsd-net@freebsd.org Subject: 10-RC unable to build kernel without INET (i.e. IPv6-only) Date: Tue, 17 Dec 2013 20:13:20 +0100 User-Agent: KMail/1.13.7 (FreeBSD/9.2-STABLE; KDE/4.10.5; amd64; ; ) MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201312172013.20436.Mark.Martinec+freebsd@ijs.si> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2013 19:13:29 -0000 Under 9.2 the following could be used to build an IPv6-only kernel: /sys/amd64/conf/TEST : include GENERIC makeoptions MKMODULESENV+="WITHOUT_INET_SUPPORT=" nooptions INET Now with stable/10 the: make buildkernel KERNCONF=TEST fails while building xen support: [...] cc -c -O2 -pipe -fno-strict-aliasing -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual - Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error- parentheses-equality -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -I/usr/src/sys/contrib/libfdt -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -mno-aes -mno-avx -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind- tables -ffreestanding -fstack-protector -Werror /usr/src/sys/dev/xen/control/control.c ctfconvert -L VERSION -g control.o cc -c -O2 -pipe -fno-strict-aliasing -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual - Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error- parentheses-equality -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/altq -I/usr/src/sys/contrib/libfdt -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -mno-aes -mno-avx -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind- tables -ffreestanding -fstack-protector -Werror /usr/src/sys/dev/xen/netback/netback.c /usr/src/sys/dev/xen/netback/netback.c:2244:8: error: use of undeclared identifier 'ifr' if (ifr->ifr_reqcap & IFCAP_TXCSUM) { ^ /usr/src/sys/dev/xen/netback/netback.c:2251:9: error: use of undeclared identifier 'ifr' if ((ifr->ifr_reqcap & IFCAP_RXCSUM)) { ^ /usr/src/sys/dev/xen/netback/netback.c:2284:18: error: use of undeclared identifier 'ifr' ifp->if_mtu = ifr->ifr_mtu; ^ /usr/src/sys/dev/xen/netback/netback.c:2292:31: error: use of undeclared identifier 'ifr' error = ifmedia_ioctl(ifp, ifr, &xnb->sc_media, cmd); ^ 4 errors generated. *** Error code 1 Stop. make[2]: stopped in /usr/obj/usr/src/sys/TEST *** Error code 1 Mark From owner-freebsd-net@FreeBSD.ORG Tue Dec 17 21:52:08 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C792F3A for ; Tue, 17 Dec 2013 21:52:08 +0000 (UTC) Received: from mail04.cae.com (mail04.cae.com [142.39.230.154]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4778010A4 for ; Tue, 17 Dec 2013 21:52:07 +0000 (UTC) Received: from caehub03.caecorp.cae.com (caehub03.caecorp.cae.com [142.39.249.107]) by mail04.cae.com (8.14.4/8.14.4) with ESMTP id rBHLp6KX006897 for ; Tue, 17 Dec 2013 16:51:06 -0500 Received: from CAEHUB02.caecorp.cae.com (142.39.20.44) by caehub03.caecorp.cae.com (142.39.249.107) with Microsoft SMTP Server (TLS) id 8.3.342.0; Tue, 17 Dec 2013 16:51:06 -0500 Received: from CAEMEX81.caecorp.cae.com ([142.39.19.46]) by caehub02.caecorp.cae.com ([142.39.20.44]) with mapi; Tue, 17 Dec 2013 16:51:06 -0500 From: Claude Marinier To: "freebsd-net@freebsd.org" Date: Tue, 17 Dec 2013 16:51:17 -0500 Subject: Re: em3 no carrier Thread-Topic: Re: em3 no carrier Thread-Index: Ac77cG1lKyg3PovkSQGQzNJMrtKdIQ== Message-ID: <34E38D657499684D9EBAA6D73CD2D1E21958EE4A62@CAEMEX81.caecorp.cae.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US X-TM-AS-Product-Ver: SMEX-10.2.0.3176-7.000.1014-20368.002 X-TM-AS-Result: No--23.458600-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2013 21:52:08 -0000 > > -----Original Message----- > > From: Rodrigo Osorio [mailto:rodrigo at bebik.net] > > Sent: Tuesday, December 17, 2013 11:40 AM > > To: Claude Marinier > > Cc: freebsd-net at freebsd.org > > Subject: Re: em3 no carrier > > > > Hi, > > > > Can you give us more informations about the network card / chips ? > > Did you found any error in /var/log/ ? > > > > - rodrigo > > My apologies, I missed that bit of information. > > Yes, there is an error in /var/log/messages. > > Dec 17 11:33:07 WANemu bird: OSPF: Socket error on em3: No buffer space a= vailable > Dec 17 11:33:47 WANemu last message repeated 4 times > Dec 17 11:35:57 WANemu last message repeated 13 times > Dec 17 11:45:57 WANemu last message repeated 60 times > Dec 17 11:55:47 WANemu last message repeated 59 times > > > Also note that the HP NC364T uses an Intel 82571EB chipset. > > > On 17/12/13 11:27 -0500, Claude Marinier wrote: >>> Hi, >>> >>> This is a WAN emulator (ipfw, DummyNet, and BIRD). FreeBSD 9.2 >>> Release (amd64) is running on DL360 G5 with a new NC364T quad-port >>> Ethernet NIC. I installed the new NIC yesterday. The four ports >>> are connected to routers in a lab. >>> >>> FreeBSD WAMemu 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898: Thu Sep 26 2= 2:50:31 UTC 2013 >>> root at bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 >>> >>> Dec 16 17:13:19 WANemu kernel: em3: >>> port 0x6020-0x603f mem 0xfdee0000-0xfdefffff,0xfde00000-0xfde7ffff irq = 17 at >>> device 0.1 on pci22 Dec 16 17:13:19 >>> WANemu kernel: em3: Using an MSI interrupt Dec 16 17:13:19 >>> WANemu kernel: em3: Ethernet address: e8:39:35:13:21:6e >>> >>> The four em interfaces are configured the same way (lines in rc.conf >>> are identical except for IP address. >>> >>> ifconfig_em3=3D"inet x.y.113.197/29 media 100baseTX mediaopt full-duple= x" >>> ifconfig_em0=3D"inet x.y.113.14/29 media 100baseTX mediaopt full-duplex= " >>> ifconfig_em2=3D"inet x.y.113.109/29 media 100baseTX mediaopt full-duple= x" >>> ifconfig_em1=3D"inet x.y.113.189/29 media 100baseTX mediaopt full-duple= x" >>> >>> The GE 0/1 port on the Cisco 2821 shows no link and ifconfig shows "no >>> carrier" for em3. Oddly, it also shows autoselect. >>> >>> em0: flags=3D8843 metric 0 mtu = 1500 >>> options=3D4019b >>> ether e8:39:35:13:21:6d >>> inet x.y.113.14 netmask 0xfffffff8 broadcast x.y.113.15 >>> inet6 fe80::ea39:35ff:fe13:216d%em0 prefixlen 64 scopeid 0x1 >>> nd6 options=3D29 >>> media: Ethernet 100baseTX >>> status: active >>> em1: flags=3D8843 metric 0 mtu = 1500 >>> options=3D4019b >>> ether e8:39:35:13:21:6c >>> inet x.y.113.189 netmask 0xfffffff8 broadcast x.y.113.191 >>> inet6 fe80::ea39:35ff:fe13:216c%em1 prefixlen 64 scopeid 0x2 >>> nd6 options=3D29 >>> media: Ethernet 100baseTX >>> status: active >>> em2: flags=3D8843 metric 0 mtu = 1500 >>> options=3D4019b >>> ether e8:39:35:13:21:6f >>> inet 131.140.113.109 netmask 0xfffffff8 broadcast 131.140.113.111 >>> inet6 fe80::ea39:35ff:fe13:216f%em2 prefixlen 64 scopeid 0x3 >>> nd6 options=3D29 >>> media: Ethernet 100baseTX >>> status: active >>> em3: flags=3D8843 metric 0 mtu = 1500 >>> options=3D4019b >>> ether e8:39:35:13:21:6e >>> inet 131.140.113.197 netmask 0xfffffff8 broadcast 131.140.113.199 >>> inet6 fe80::ea39:35ff:fe13:216e%em3 prefixlen 64 scopeid 0x4 >>> nd6 options=3D29 >>> media: Ethernet 100baseTX (autoselect) >>> status: no carrier >>> >>> I have tried three Ethernet cables with no difference. This morning, >>> I briefly booted Puppy Linux from USB and manually configured the four >>> Ethernet interfaces. They all work (ping good to all four routers); >>> the Cisco on em3 shows happy lights. I then booted back into FreeBSD, >>> the problem with em3 remains. I have not had any success searching with >>> Google; I may not have used suitable search terms. >>> >>> One more odd symptom: bmon has trouble with these interfaces. >>> >>> em0=E895^S!m on WANemu >>> Name >>> qqqqqqqqqqqqqqqqqqqqqqqq >>> WANemu (local) >>> 0 em0=E895^S!m >>> 1 em1=E895^S!l >>> 2 em2=E895^S!o >>> 3 em3=E895^S!n >>> 4 usbus0 >>> 5 usbus1 >>> 6 usbus2 >>> 7 usbus3 >>> 8 usbus4 >>> 9 usbus5 >>> 10 ipfw0 >>> 11 lo0 >>> >>> It looks like FreeBSD is somehow mishandling these interfaces. Before >>> purchasing it, I searched with Google and found reports of people using >>> this device without problems. I do not know how to proceed. >>> >>> Thank you for your help. Oddly, replacing em3 with bce1 left the system with only two functionning interfaces. The order of the interface definitions in rc.local makes a difference. The worst order is bce1, em0, em2, em1 which left only em0 and em2 working. Changing the order to bce1, em0, em1, em2 allows the three 'em' interfaces to work. In both, bce1 is not working (no carrier). Changing the order to em0, em1, em2, bce1 did not improve the situation. This is a WAN emulator and the rc.conf definitions are generated by the configuration program from site names, hence the odd order. I am changing the order manually. I have reconfigured the Cisco router to use G0/0 instead of G0/1. This has made no noticeable difference. Does anyone know how to order the definitions? -- C Marinier From owner-freebsd-net@FreeBSD.ORG Tue Dec 17 22:24:14 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 35D24AEE for ; Tue, 17 Dec 2013 22:24:14 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 469BD1328 for ; Tue, 17 Dec 2013 22:24:13 +0000 (UTC) Received: from [192.168.1.200] (p508F1D30.dip0.t-ipconnect.de [80.143.29.48]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id BFDE41C0C069C; Tue, 17 Dec 2013 23:24:09 +0100 (CET) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) Subject: Re: em3 no carrier From: Michael Tuexen In-Reply-To: <34E38D657499684D9EBAA6D73CD2D1E21958EE4A62@CAEMEX81.caecorp.cae.com> Date: Tue, 17 Dec 2013 23:24:09 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <34E38D657499684D9EBAA6D73CD2D1E21958EE4A62@CAEMEX81.caecorp.cae.com> To: Claude Marinier X-Mailer: Apple Mail (2.1510) Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Dec 2013 22:24:14 -0000 On Dec 17, 2013, at 10:51 PM, Claude Marinier = wrote: >=20 >>> -----Original Message----- >>> From: Rodrigo Osorio [mailto:rodrigo at bebik.net] >>> Sent: Tuesday, December 17, 2013 11:40 AM >>> To: Claude Marinier >>> Cc: freebsd-net at freebsd.org >>> Subject: Re: em3 no carrier >>>=20 >>> Hi, >>>=20 >>> Can you give us more informations about the network card / chips ? >>> Did you found any error in /var/log/ ? >>>=20 >>> - rodrigo >>=20 >> My apologies, I missed that bit of information. >>=20 >> Yes, there is an error in /var/log/messages. >>=20 >> Dec 17 11:33:07 WANemu bird: OSPF: Socket error on em3: No buffer = space available >> Dec 17 11:33:47 WANemu last message repeated 4 times >> Dec 17 11:35:57 WANemu last message repeated 13 times >> Dec 17 11:45:57 WANemu last message repeated 60 times >> Dec 17 11:55:47 WANemu last message repeated 59 times >>=20 >>=20 >> Also note that the HP NC364T uses an Intel 82571EB chipset. >>=20 >>> On 17/12/13 11:27 -0500, Claude Marinier wrote: >>>> Hi, >>>>=20 >>>> This is a WAN emulator (ipfw, DummyNet, and BIRD). FreeBSD 9.2 >>>> Release (amd64) is running on DL360 G5 with a new NC364T quad-port >>>> Ethernet NIC. I installed the new NIC yesterday. The four ports >>>> are connected to routers in a lab. >>>>=20 >>>> FreeBSD WAMemu 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898: Thu Sep = 26 22:50:31 UTC 2013 >>>> root at bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 >>>>=20 >>>> Dec 16 17:13:19 WANemu kernel: em3: >>>> port 0x6020-0x603f mem 0xfdee0000-0xfdefffff,0xfde00000-0xfde7ffff = irq 17 at >>>> device 0.1 on pci22 Dec 16 17:13:19 >>>> WANemu kernel: em3: Using an MSI interrupt Dec 16 17:13:19 >>>> WANemu kernel: em3: Ethernet address: e8:39:35:13:21:6e >>>>=20 >>>> The four em interfaces are configured the same way (lines in = rc.conf >>>> are identical except for IP address. >>>>=20 >>>> ifconfig_em3=3D"inet x.y.113.197/29 media 100baseTX mediaopt = full-duplex" >>>> ifconfig_em0=3D"inet x.y.113.14/29 media 100baseTX mediaopt = full-duplex" >>>> ifconfig_em2=3D"inet x.y.113.109/29 media 100baseTX mediaopt = full-duplex" >>>> ifconfig_em1=3D"inet x.y.113.189/29 media 100baseTX mediaopt = full-duplex" >>>>=20 >>>> The GE 0/1 port on the Cisco 2821 shows no link and ifconfig shows = "no >>>> carrier" for em3. Oddly, it also shows autoselect. >>>>=20 >>>> em0: flags=3D8843 metric 0 = mtu 1500 >>>> = options=3D4019b >>>> ether e8:39:35:13:21:6d >>>> inet x.y.113.14 netmask 0xfffffff8 broadcast x.y.113.15 >>>> inet6 fe80::ea39:35ff:fe13:216d%em0 prefixlen 64 scopeid 0x1 >>>> nd6 options=3D29 >>>> media: Ethernet 100baseTX >>>> status: active >>>> em1: flags=3D8843 metric 0 = mtu 1500 >>>> = options=3D4019b >>>> ether e8:39:35:13:21:6c >>>> inet x.y.113.189 netmask 0xfffffff8 broadcast x.y.113.191 >>>> inet6 fe80::ea39:35ff:fe13:216c%em1 prefixlen 64 scopeid 0x2 >>>> nd6 options=3D29 >>>> media: Ethernet 100baseTX >>>> status: active >>>> em2: flags=3D8843 metric 0 = mtu 1500 >>>> = options=3D4019b >>>> ether e8:39:35:13:21:6f >>>> inet 131.140.113.109 netmask 0xfffffff8 broadcast = 131.140.113.111 >>>> inet6 fe80::ea39:35ff:fe13:216f%em2 prefixlen 64 scopeid 0x3 >>>> nd6 options=3D29 >>>> media: Ethernet 100baseTX >>>> status: active >>>> em3: flags=3D8843 metric 0 = mtu 1500 >>>> = options=3D4019b >>>> ether e8:39:35:13:21:6e >>>> inet 131.140.113.197 netmask 0xfffffff8 broadcast = 131.140.113.199 >>>> inet6 fe80::ea39:35ff:fe13:216e%em3 prefixlen 64 scopeid 0x4 >>>> nd6 options=3D29 >>>> media: Ethernet 100baseTX (autoselect) >>>> status: no carrier >>>>=20 >>>> I have tried three Ethernet cables with no difference. This = morning, >>>> I briefly booted Puppy Linux from USB and manually configured the = four >>>> Ethernet interfaces. They all work (ping good to all four routers); >>>> the Cisco on em3 shows happy lights. I then booted back into = FreeBSD, >>>> the problem with em3 remains. I have not had any success searching = with >>>> Google; I may not have used suitable search terms. >>>>=20 >>>> One more odd symptom: bmon has trouble with these interfaces. >>>>=20 >>>> em0=E895^S!m on WANemu >>>> Name >>>> qqqqqqqqqqqqqqqqqqqqqqqq >>>> WANemu (local) >>>> 0 em0=E895^S!m >>>> 1 em1=E895^S!l >>>> 2 em2=E895^S!o >>>> 3 em3=E895^S!n >>>> 4 usbus0 >>>> 5 usbus1 >>>> 6 usbus2 >>>> 7 usbus3 >>>> 8 usbus4 >>>> 9 usbus5 >>>> 10 ipfw0 >>>> 11 lo0 >>>>=20 >>>> It looks like FreeBSD is somehow mishandling these interfaces. = Before >>>> purchasing it, I searched with Google and found reports of people = using >>>> this device without problems. I do not know how to proceed. >>>>=20 >>>> Thank you for your help. >=20 > Oddly, replacing em3 with bce1 left the system with only two = functionning > interfaces. The order of the interface definitions in rc.local makes > a difference. The worst order is bce1, em0, em2, em1 which left only > em0 and em2 working. Changing the order to bce1, em0, em1, em2 allows > the three 'em' interfaces to work. In both, bce1 is not working (no > carrier). Changing the order to em0, em1, em2, bce1 did not improve > the situation. >=20 > This is a WAN emulator and the rc.conf definitions are generated by = the > configuration program from site names, hence the odd order. I am = changing > the order manually. >=20 > I have reconfigured the Cisco router to use G0/0 instead of G0/1. This > has made no noticeable difference. Did you configure it to 100MBit, full-duplex? Best regards Michael >=20 > Does anyone know how to order the definitions? >=20 > -- > C Marinier >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >=20 From owner-freebsd-net@FreeBSD.ORG Wed Dec 18 00:43:33 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3488AC8B for ; Wed, 18 Dec 2013 00:43:33 +0000 (UTC) Received: from mail01.cae.com (mail01.cae.com [142.39.230.151]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F40F41E58 for ; Wed, 18 Dec 2013 00:43:32 +0000 (UTC) Received: from caehub03.caecorp.cae.com (caehub03.caecorp.cae.com [142.39.249.107]) by mail01.cae.com (8.14.4/8.14.4) with ESMTP id rBI0h7tZ021593; Tue, 17 Dec 2013 19:43:07 -0500 Received: from CAEHUB02.caecorp.cae.com (142.39.20.44) by caehub03.caecorp.cae.com (142.39.249.107) with Microsoft SMTP Server (TLS) id 8.3.342.0; Tue, 17 Dec 2013 19:43:07 -0500 Received: from CAEMEX81.caecorp.cae.com ([142.39.19.46]) by caehub02.caecorp.cae.com ([142.39.20.44]) with mapi; Tue, 17 Dec 2013 19:43:07 -0500 From: Claude Marinier To: Michael Tuexen Date: Tue, 17 Dec 2013 19:43:05 -0500 Subject: RE: em3 no carrier Thread-Topic: em3 no carrier Thread-Index: Ac77dsRXYsuIp2jpT2C+8GnzrXdvjwAEtRBA Message-ID: <34E38D657499684D9EBAA6D73CD2D1E21958EE4AD3@CAEMEX81.caecorp.cae.com> References: <34E38D657499684D9EBAA6D73CD2D1E21958EE4A62@CAEMEX81.caecorp.cae.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US X-TM-AS-Product-Ver: SMEX-10.2.0.3176-7.000.1014-20368.003 X-TM-AS-Result: No--11.901600-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2013 00:43:33 -0000 -----Original Message----- From: Michael Tuexen [mailto:Michael.Tuexen@lurchi.franken.de]=20 Sent: Tuesday, December 17, 2013 5:24 PM To: Claude Marinier Cc: freebsd-net@freebsd.org Subject: Re: em3 no carrier On Dec 17, 2013, at 10:51 PM, Claude Marinier wro= te: >=20 >>> -----Original Message----- >>> From: Rodrigo Osorio [mailto:rodrigo at bebik.net] >>> Sent: Tuesday, December 17, 2013 11:40 AM >>> To: Claude Marinier >>> Cc: freebsd-net at freebsd.org >>> Subject: Re: em3 no carrier >>>=20 >>> Hi, >>>=20 >>> Can you give us more informations about the network card / chips ? >>> Did you found any error in /var/log/ ? >>>=20 >>> - rodrigo >>=20 >> My apologies, I missed that bit of information. >>=20 >> Yes, there is an error in /var/log/messages. >>=20 >> Dec 17 11:33:07 WANemu bird: OSPF: Socket error on em3: No buffer=20 >> space available Dec 17 11:33:47 WANemu last message repeated 4 times=20 >> Dec 17 11:35:57 WANemu last message repeated 13 times Dec 17 11:45:57=20 >> WANemu last message repeated 60 times Dec 17 11:55:47 WANemu last=20 >> message repeated 59 times >>=20 >>=20 >> Also note that the HP NC364T uses an Intel 82571EB chipset. >>=20 >>> On 17/12/13 11:27 -0500, Claude Marinier wrote: >>>> Hi, >>>>=20 >>>> This is a WAN emulator (ipfw, DummyNet, and BIRD). FreeBSD 9.2=20 >>>> Release (amd64) is running on DL360 G5 with a new NC364T quad-port=20 >>>> Ethernet NIC. I installed the new NIC yesterday. The four ports are=20 >>>> connected to routers in a lab. >>>>=20 >>>> FreeBSD WAMemu 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898: Thu Sep=20 >>>> 26 22:50:31 UTC 2013 root at=20 >>>> bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 >>>>=20 >>>> Dec 16 17:13:19 WANemu kernel: em3: >>> Connection 7.3.8> port 0x6020-0x603f mem=20 >>>> 0xfdee0000-0xfdefffff,0xfde00000-0xfde7ffff irq 17 at device 0.1 on=20 >>>> pci22 Dec 16 17:13:19 WANemu kernel: em3: Using an MSI interrupt=20 >>>> Dec 16 17:13:19 WANemu kernel: em3: Ethernet address:=20 >>>> e8:39:35:13:21:6e >>>>=20 >>>> The four em interfaces are configured the same way (lines in=20 >>>> rc.conf are identical except for IP address. >>>>=20 >>>> ifconfig_em3=3D"inet x.y.113.197/29 media 100baseTX mediaopt full-dupl= ex" >>>> ifconfig_em0=3D"inet x.y.113.14/29 media 100baseTX mediaopt full-duple= x" >>>> ifconfig_em2=3D"inet x.y.113.109/29 media 100baseTX mediaopt full-dupl= ex" >>>> ifconfig_em1=3D"inet x.y.113.189/29 media 100baseTX mediaopt full-dupl= ex" >>>>=20 >>>> The GE 0/1 port on the Cisco 2821 shows no link and ifconfig shows=20 >>>> "no carrier" for em3. Oddly, it also shows autoselect. >>>>=20 >>>> em0: flags=3D8843 metric 0 mtu= 1500 >>>> options=3D4019b >>>> ether e8:39:35:13:21:6d >>>> inet x.y.113.14 netmask 0xfffffff8 broadcast x.y.113.15 >>>> inet6 fe80::ea39:35ff:fe13:216d%em0 prefixlen 64 scopeid 0x1 >>>> nd6 options=3D29 >>>> media: Ethernet 100baseTX >>>> status: active >>>> em1: flags=3D8843 metric 0 mtu= 1500 >>>> options=3D4019b >>>> ether e8:39:35:13:21:6c >>>> inet x.y.113.189 netmask 0xfffffff8 broadcast x.y.113.191 >>>> inet6 fe80::ea39:35ff:fe13:216c%em1 prefixlen 64 scopeid 0x2 >>>> nd6 options=3D29 >>>> media: Ethernet 100baseTX >>>> status: active >>>> em2: flags=3D8843 metric 0 mtu= 1500 >>>> options=3D4019b >>>> ether e8:39:35:13:21:6f >>>> inet 131.140.113.109 netmask 0xfffffff8 broadcast 131.140.113.111 >>>> inet6 fe80::ea39:35ff:fe13:216f%em2 prefixlen 64 scopeid 0x3 >>>> nd6 options=3D29 >>>> media: Ethernet 100baseTX >>>> status: active >>>> em3: flags=3D8843 metric 0 mtu= 1500 >>>> options=3D4019b >>>> ether e8:39:35:13:21:6e >>>> inet 131.140.113.197 netmask 0xfffffff8 broadcast 131.140.113.199 >>>> inet6 fe80::ea39:35ff:fe13:216e%em3 prefixlen 64 scopeid 0x4 >>>> nd6 options=3D29 >>>> media: Ethernet 100baseTX (autoselect) >>>> status: no carrier >>>>=20 >>>> I have tried three Ethernet cables with no difference. This=20 >>>> morning, I briefly booted Puppy Linux from USB and manually=20 >>>> configured the four Ethernet interfaces. They all work (ping good=20 >>>> to all four routers); the Cisco on em3 shows happy lights. I then=20 >>>> booted back into FreeBSD, the problem with em3 remains. I have not=20 >>>> had any success searching with Google; I may not have used suitable se= arch terms. >>>>=20 >>>> One more odd symptom: bmon has trouble with these interfaces. >>>>=20 >>>> em0=E895^S!m on WANemu >>>> Name >>>> qqqqqqqqqqqqqqqqqqqqqqqq >>>> WANemu (local) >>>> 0 em0=E895^S!m >>>> 1 em1=E895^S!l >>>> 2 em2=E895^S!o >>>> 3 em3=E895^S!n >>>> 4 usbus0 >>>> 5 usbus1 >>>> 6 usbus2 >>>> 7 usbus3 >>>> 8 usbus4 >>>> 9 usbus5 >>>> 10 ipfw0 >>>> 11 lo0 >>>>=20 >>>> It looks like FreeBSD is somehow mishandling these interfaces.=20 >>>> Before purchasing it, I searched with Google and found reports of=20 >>>> people using this device without problems. I do not know how to procee= d. >>>>=20 >>>> Thank you for your help. >=20 > Oddly, replacing em3 with bce1 left the system with only two=20 > functionning interfaces. The order of the interface definitions in=20 > rc.local makes a difference. The worst order is bce1, em0, em2, em1=20 > which left only > em0 and em2 working. Changing the order to bce1, em0, em1, em2 allows=20 > the three 'em' interfaces to work. In both, bce1 is not working (no=20 > carrier). Changing the order to em0, em1, em2, bce1 did not improve=20 > the situation. >=20 > This is a WAN emulator and the rc.conf definitions are generated by=20 > the configuration program from site names, hence the odd order. I am=20 > changing the order manually. >=20 > I have reconfigured the Cisco router to use G0/0 instead of G0/1. This=20 > has made no noticeable difference. Did you configure it to 100MBit, full-duplex? ------- Yes, I configured everything (all eight interfaces: four on FreeBSD and one= on each of the routers) to be 100baseTX and full-duplex. As a test, I left out the 'media' and 'mediaopt' portions of the interface = definitions in rc.conf and all the interfaces came up but three were in hal= f-duplex mode, I think em0 was full-duplex (I am not in the lab to check). I have heard horror stories about Ethernet auto sensing, so I make sure the= Cisco routers and FreeBSD specify 100 Mbps and full-duplex. I begin to won= der if even that is safe. :-( [ why does Outlook not quote properly ] From owner-freebsd-net@FreeBSD.ORG Wed Dec 18 01:03:55 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 032B86A6 for ; Wed, 18 Dec 2013 01:03:55 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AEAD51FA9 for ; Wed, 18 Dec 2013 01:03:54 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id rBI13hOX003080 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 17 Dec 2013 17:03:43 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id rBI13hRF003079; Tue, 17 Dec 2013 17:03:43 -0800 (PST) (envelope-from jmg) Date: Tue, 17 Dec 2013 17:03:42 -0800 From: John-Mark Gurney To: Claude Marinier Subject: Re: em3 no carrier Message-ID: <20131218010342.GG99167@funkthat.com> Mail-Followup-To: Claude Marinier , Michael Tuexen , "freebsd-net@freebsd.org" References: <34E38D657499684D9EBAA6D73CD2D1E21958EE4A62@CAEMEX81.caecorp.cae.com> <34E38D657499684D9EBAA6D73CD2D1E21958EE4AD3@CAEMEX81.caecorp.cae.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <34E38D657499684D9EBAA6D73CD2D1E21958EE4AD3@CAEMEX81.caecorp.cae.com> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Tue, 17 Dec 2013 17:03:43 -0800 (PST) Cc: Michael Tuexen , "freebsd-net@freebsd.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2013 01:03:55 -0000 Claude Marinier wrote this message on Tue, Dec 17, 2013 at 19:43 -0500: > -----Original Message----- > From: Michael Tuexen [mailto:Michael.Tuexen@lurchi.franken.de] > Sent: Tuesday, December 17, 2013 5:24 PM > To: Claude Marinier > Cc: freebsd-net@freebsd.org > Subject: Re: em3 no carrier > > On Dec 17, 2013, at 10:51 PM, Claude Marinier wrote: > > > > >>> -----Original Message----- > >>> From: Rodrigo Osorio [mailto:rodrigo at bebik.net] > >>> Sent: Tuesday, December 17, 2013 11:40 AM > >>> To: Claude Marinier > >>> Cc: freebsd-net at freebsd.org > >>> Subject: Re: em3 no carrier > >>> > >>> Hi, > >>> > >>> Can you give us more informations about the network card / chips ? > >>> Did you found any error in /var/log/ ? > >>> > >>> - rodrigo > >> > >> My apologies, I missed that bit of information. > >> > >> Yes, there is an error in /var/log/messages. > >> > >> Dec 17 11:33:07 WANemu bird: OSPF: Socket error on em3: No buffer > >> space available Dec 17 11:33:47 WANemu last message repeated 4 times > >> Dec 17 11:35:57 WANemu last message repeated 13 times Dec 17 11:45:57 > >> WANemu last message repeated 60 times Dec 17 11:55:47 WANemu last > >> message repeated 59 times > >> > >> > >> Also note that the HP NC364T uses an Intel 82571EB chipset. > >> > >>> On 17/12/13 11:27 -0500, Claude Marinier wrote: > >>>> Hi, > >>>> > >>>> This is a WAN emulator (ipfw, DummyNet, and BIRD). FreeBSD 9.2 > >>>> Release (amd64) is running on DL360 G5 with a new NC364T quad-port > >>>> Ethernet NIC. I installed the new NIC yesterday. The four ports are > >>>> connected to routers in a lab. > >>>> > >>>> FreeBSD WAMemu 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898: Thu Sep > >>>> 26 22:50:31 UTC 2013 root at > >>>> bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > >>>> > >>>> Dec 16 17:13:19 WANemu kernel: em3: >>>> Connection 7.3.8> port 0x6020-0x603f mem > >>>> 0xfdee0000-0xfdefffff,0xfde00000-0xfde7ffff irq 17 at device 0.1 on > >>>> pci22 Dec 16 17:13:19 WANemu kernel: em3: Using an MSI interrupt > >>>> Dec 16 17:13:19 WANemu kernel: em3: Ethernet address: > >>>> e8:39:35:13:21:6e > >>>> > >>>> The four em interfaces are configured the same way (lines in > >>>> rc.conf are identical except for IP address. > >>>> > >>>> ifconfig_em3="inet x.y.113.197/29 media 100baseTX mediaopt full-duplex" > >>>> ifconfig_em0="inet x.y.113.14/29 media 100baseTX mediaopt full-duplex" > >>>> ifconfig_em2="inet x.y.113.109/29 media 100baseTX mediaopt full-duplex" > >>>> ifconfig_em1="inet x.y.113.189/29 media 100baseTX mediaopt full-duplex" > >>>> > >>>> The GE 0/1 port on the Cisco 2821 shows no link and ifconfig shows > >>>> "no carrier" for em3. Oddly, it also shows autoselect. > >>>> > >>>> em0: flags=8843 metric 0 mtu 1500 > >>>> options=4019b > >>>> ether e8:39:35:13:21:6d > >>>> inet x.y.113.14 netmask 0xfffffff8 broadcast x.y.113.15 > >>>> inet6 fe80::ea39:35ff:fe13:216d%em0 prefixlen 64 scopeid 0x1 > >>>> nd6 options=29 > >>>> media: Ethernet 100baseTX > >>>> status: active > >>>> em1: flags=8843 metric 0 mtu 1500 > >>>> options=4019b > >>>> ether e8:39:35:13:21:6c > >>>> inet x.y.113.189 netmask 0xfffffff8 broadcast x.y.113.191 > >>>> inet6 fe80::ea39:35ff:fe13:216c%em1 prefixlen 64 scopeid 0x2 > >>>> nd6 options=29 > >>>> media: Ethernet 100baseTX > >>>> status: active > >>>> em2: flags=8843 metric 0 mtu 1500 > >>>> options=4019b > >>>> ether e8:39:35:13:21:6f > >>>> inet 131.140.113.109 netmask 0xfffffff8 broadcast 131.140.113.111 > >>>> inet6 fe80::ea39:35ff:fe13:216f%em2 prefixlen 64 scopeid 0x3 > >>>> nd6 options=29 > >>>> media: Ethernet 100baseTX > >>>> status: active > >>>> em3: flags=8843 metric 0 mtu 1500 > >>>> options=4019b > >>>> ether e8:39:35:13:21:6e > >>>> inet 131.140.113.197 netmask 0xfffffff8 broadcast 131.140.113.199 > >>>> inet6 fe80::ea39:35ff:fe13:216e%em3 prefixlen 64 scopeid 0x4 > >>>> nd6 options=29 > >>>> media: Ethernet 100baseTX (autoselect) > >>>> status: no carrier > >>>> > >>>> I have tried three Ethernet cables with no difference. This > >>>> morning, I briefly booted Puppy Linux from USB and manually > >>>> configured the four Ethernet interfaces. They all work (ping good > >>>> to all four routers); the Cisco on em3 shows happy lights. I then > >>>> booted back into FreeBSD, the problem with em3 remains. I have not > >>>> had any success searching with Google; I may not have used suitable search terms. > >>>> > >>>> One more odd symptom: bmon has trouble with these interfaces. > >>>> > >>>> em0č95^S!m on WANemu > >>>> Name > >>>> qqqqqqqqqqqqqqqqqqqqqqqq > >>>> WANemu (local) > >>>> 0 em0č95^S!m > >>>> 1 em1č95^S!l > >>>> 2 em2č95^S!o > >>>> 3 em3č95^S!n > >>>> 4 usbus0 > >>>> 5 usbus1 > >>>> 6 usbus2 > >>>> 7 usbus3 > >>>> 8 usbus4 > >>>> 9 usbus5 > >>>> 10 ipfw0 > >>>> 11 lo0 > >>>> > >>>> It looks like FreeBSD is somehow mishandling these interfaces. > >>>> Before purchasing it, I searched with Google and found reports of > >>>> people using this device without problems. I do not know how to proceed. > >>>> > >>>> Thank you for your help. > > > > Oddly, replacing em3 with bce1 left the system with only two > > functionning interfaces. The order of the interface definitions in > > rc.local makes a difference. The worst order is bce1, em0, em2, em1 > > which left only > > em0 and em2 working. Changing the order to bce1, em0, em1, em2 allows > > the three 'em' interfaces to work. In both, bce1 is not working (no > > carrier). Changing the order to em0, em1, em2, bce1 did not improve > > the situation. > > > > This is a WAN emulator and the rc.conf definitions are generated by > > the configuration program from site names, hence the odd order. I am > > changing the order manually. > > > > I have reconfigured the Cisco router to use G0/0 instead of G0/1. This > > has made no noticeable difference. > > Did you configure it to 100MBit, full-duplex? > > ------- > > Yes, I configured everything (all eight interfaces: four on FreeBSD and one on each of the routers) to be 100baseTX and full-duplex. > > As a test, I left out the 'media' and 'mediaopt' portions of the interface definitions in rc.conf and all the interfaces came up but three were in half-duplex mode, I think em0 was full-duplex (I am not in the lab to check). > > I have heard horror stories about Ethernet auto sensing, so I make sure the Cisco routers and FreeBSD specify 100 Mbps and full-duplex. I begin to wonder if even that is safe. :-( > > [ why does Outlook not quote properly ] It could be that there is a race condition when the media opts are set at the same time the interface is brought up... I have you tried to down the interface, set the media opts, wait a second or two, and then bring up the interface? I would go back to autoneg... Even though the 2821 is old, the autoneg issue was mostly when both sides were playing bad, and your nics are modern... At a previous company, we had an issue w/ our box at a client site... The box would work fine for months, and then for some reason it would suffer packet loss... After a long bit of puzzling, (I don't even remeber how we figured it out, but I think we got a dump of the switch configuration), it turned out that someone had configured our box to be 100mbit/full duplex, but not the switch... It turned out that this was "ok" for fxp as it would autoneg when brought up, and then it would switch to 100mbit/full... Both sides would be happy till the switch rebooted and then tried to autoneg again, but now the fxp had disabled it, and the switch would fall back to 100mbit/half... I fixed fxp to down/up the phy when setting fixed modes so that the switch would loose it's autoneg and then the customer would immediately see the issue instead of weeks/months/years later when the switch was power cycled... after the above, I considered anything not autoneg to be bad... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Wed Dec 18 02:13:31 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A030F4C for ; Wed, 18 Dec 2013 02:13:31 +0000 (UTC) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B915A149D for ; Wed, 18 Dec 2013 02:13:30 +0000 (UTC) Received: from r2d2 ([82.69.179.241]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50007133480.msg for ; Wed, 18 Dec 2013 02:13:21 +0000 X-Spam-Processed: mail1.multiplay.co.uk, Wed, 18 Dec 2013 02:13:21 +0000 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.179.241 X-Return-Path: prvs=106493f26b=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-net@freebsd.org Message-ID: From: "Steven Hartland" To: "John-Mark Gurney" , "Claude Marinier" References: <34E38D657499684D9EBAA6D73CD2D1E21958EE4A62@CAEMEX81.caecorp.cae.com> <34E38D657499684D9EBAA6D73CD2D1E21958EE4AD3@CAEMEX81.caecorp.cae.com> <20131218010342.GG99167@funkthat.com> Subject: Re: em3 no carrier Date: Wed, 18 Dec 2013 02:13:23 -0000 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: Michael Tuexen , freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2013 02:13:31 -0000 ----- Original Message ----- From: "John-Mark Gurney" To: "Claude Marinier" Cc: "Michael Tuexen" ; Sent: Wednesday, December 18, 2013 1:03 AM Subject: Re: em3 no carrier Claude Marinier wrote this message on Tue, Dec 17, 2013 at 19:43 -0500: > -----Original Message----- > From: Michael Tuexen [mailto:Michael.Tuexen@lurchi.franken.de] > Sent: Tuesday, December 17, 2013 5:24 PM > To: Claude Marinier > Cc: freebsd-net@freebsd.org > Subject: Re: em3 no carrier > > On Dec 17, 2013, at 10:51 PM, Claude Marinier wrote: > > > > >>> -----Original Message----- > >>> From: Rodrigo Osorio [mailto:rodrigo at bebik.net] > >>> Sent: Tuesday, December 17, 2013 11:40 AM > >>> To: Claude Marinier > >>> Cc: freebsd-net at freebsd.org > >>> Subject: Re: em3 no carrier > >>> > >>> Hi, > >>> > >>> Can you give us more informations about the network card / chips ? > >>> Did you found any error in /var/log/ ? > >>> > >>> - rodrigo > >> > >> My apologies, I missed that bit of information. > >> > >> Yes, there is an error in /var/log/messages. > >> > >> Dec 17 11:33:07 WANemu bird: OSPF: Socket error on em3: No buffer > >> space available Dec 17 11:33:47 WANemu last message repeated 4 times > >> Dec 17 11:35:57 WANemu last message repeated 13 times Dec 17 11:45:57 > >> WANemu last message repeated 60 times Dec 17 11:55:47 WANemu last > >> message repeated 59 times > >> > >> > >> Also note that the HP NC364T uses an Intel 82571EB chipset. > >> > >>> On 17/12/13 11:27 -0500, Claude Marinier wrote: > >>>> Hi, > >>>> > >>>> This is a WAN emulator (ipfw, DummyNet, and BIRD). FreeBSD 9.2 > >>>> Release (amd64) is running on DL360 G5 with a new NC364T quad-port > >>>> Ethernet NIC. I installed the new NIC yesterday. The four ports are > >>>> connected to routers in a lab. > >>>> > >>>> FreeBSD WAMemu 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898: Thu Sep > >>>> 26 22:50:31 UTC 2013 root at > >>>> bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > >>>> > >>>> Dec 16 17:13:19 WANemu kernel: em3: >>>> Connection 7.3.8> port 0x6020-0x603f mem > >>>> 0xfdee0000-0xfdefffff,0xfde00000-0xfde7ffff irq 17 at device 0.1 on > >>>> pci22 Dec 16 17:13:19 WANemu kernel: em3: Using an MSI interrupt > >>>> Dec 16 17:13:19 WANemu kernel: em3: Ethernet address: > >>>> e8:39:35:13:21:6e > >>>> > >>>> The four em interfaces are configured the same way (lines in > >>>> rc.conf are identical except for IP address. > >>>> > >>>> ifconfig_em3="inet x.y.113.197/29 media 100baseTX mediaopt full-duplex" > >>>> ifconfig_em0="inet x.y.113.14/29 media 100baseTX mediaopt full-duplex" > >>>> ifconfig_em2="inet x.y.113.109/29 media 100baseTX mediaopt full-duplex" > >>>> ifconfig_em1="inet x.y.113.189/29 media 100baseTX mediaopt full-duplex" > >>>> > >>>> The GE 0/1 port on the Cisco 2821 shows no link and ifconfig shows > >>>> "no carrier" for em3. Oddly, it also shows autoselect. > >>>> > >>>> em0: flags=8843 metric 0 mtu 1500 > >>>> options=4019b > >>>> ether e8:39:35:13:21:6d > >>>> inet x.y.113.14 netmask 0xfffffff8 broadcast x.y.113.15 > >>>> inet6 fe80::ea39:35ff:fe13:216d%em0 prefixlen 64 scopeid 0x1 > >>>> nd6 options=29 > >>>> media: Ethernet 100baseTX > >>>> status: active > >>>> em1: flags=8843 metric 0 mtu 1500 > >>>> options=4019b > >>>> ether e8:39:35:13:21:6c > >>>> inet x.y.113.189 netmask 0xfffffff8 broadcast x.y.113.191 > >>>> inet6 fe80::ea39:35ff:fe13:216c%em1 prefixlen 64 scopeid 0x2 > >>>> nd6 options=29 > >>>> media: Ethernet 100baseTX > >>>> status: active > >>>> em2: flags=8843 metric 0 mtu 1500 > >>>> options=4019b > >>>> ether e8:39:35:13:21:6f > >>>> inet 131.140.113.109 netmask 0xfffffff8 broadcast 131.140.113.111 > >>>> inet6 fe80::ea39:35ff:fe13:216f%em2 prefixlen 64 scopeid 0x3 > >>>> nd6 options=29 > >>>> media: Ethernet 100baseTX > >>>> status: active > >>>> em3: flags=8843 metric 0 mtu 1500 > >>>> options=4019b > >>>> ether e8:39:35:13:21:6e > >>>> inet 131.140.113.197 netmask 0xfffffff8 broadcast 131.140.113.199 > >>>> inet6 fe80::ea39:35ff:fe13:216e%em3 prefixlen 64 scopeid 0x4 > >>>> nd6 options=29 > >>>> media: Ethernet 100baseTX (autoselect) > >>>> status: no carrier > >>>> > >>>> I have tried three Ethernet cables with no difference. This > >>>> morning, I briefly booted Puppy Linux from USB and manually > >>>> configured the four Ethernet interfaces. They all work (ping good > >>>> to all four routers); the Cisco on em3 shows happy lights. I then > >>>> booted back into FreeBSD, the problem with em3 remains. I have not > >>>> had any success searching with Google; I may not have used suitable search terms. > >>>> > >>>> One more odd symptom: bmon has trouble with these interfaces. > >>>> > >>>> em0č95^S!m on WANemu > >>>> Name > >>>> qqqqqqqqqqqqqqqqqqqqqqqq > >>>> WANemu (local) > >>>> 0 em0č95^S!m > >>>> 1 em1č95^S!l > >>>> 2 em2č95^S!o > >>>> 3 em3č95^S!n > >>>> 4 usbus0 > >>>> 5 usbus1 > >>>> 6 usbus2 > >>>> 7 usbus3 > >>>> 8 usbus4 > >>>> 9 usbus5 > >>>> 10 ipfw0 > >>>> 11 lo0 > >>>> > >>>> It looks like FreeBSD is somehow mishandling these interfaces. > >>>> Before purchasing it, I searched with Google and found reports of > >>>> people using this device without problems. I do not know how to proceed. > >>>> > >>>> Thank you for your help. > > > > Oddly, replacing em3 with bce1 left the system with only two > > functionning interfaces. The order of the interface definitions in > > rc.local makes a difference. The worst order is bce1, em0, em2, em1 > > which left only > > em0 and em2 working. Changing the order to bce1, em0, em1, em2 allows > > the three 'em' interfaces to work. In both, bce1 is not working (no > > carrier). Changing the order to em0, em1, em2, bce1 did not improve > > the situation. Silly question but your not running out of mbufs on init are you? I've seen a very similar issues before and increasing mbufs and in some cases jumbos fixed the issue. I would also strongly avoid using hard coded speed / duplex and if you do need to use it ensure both ends are hard coded otherwise it just wont work. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-net@FreeBSD.ORG Wed Dec 18 03:37:45 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8E893DD3; Wed, 18 Dec 2013 03:37:45 +0000 (UTC) Received: from mail-oa0-x22f.google.com (mail-oa0-x22f.google.com [IPv6:2607:f8b0:4003:c02::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 395671D68; Wed, 18 Dec 2013 03:37:45 +0000 (UTC) Received: by mail-oa0-f47.google.com with SMTP id k1so7594606oag.20 for ; Tue, 17 Dec 2013 19:37:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=OqXjrfWmjont4SBzSuT9a2wyKqNsktacfd/mxqddxTA=; b=sBVjOe+nDlz32+2nmn2eu2LDzay1ig3IS0/zVF+KKtJAVAPynupkFkZldspUB58daA +VwpCfpUAt943rXok50Xvmowy6IrhjIBuSPrcnqPcwjbtxxU79O6OIq2C+nL/wi95GYs EdQ/Cjxe5vJqzbptM0UTXM2U+oc4wUhihipLyoGVYK0jrRcxYIh6CN9FtN2/ekIkN+5T uJ7RXN53TunI+3WhJ1GKn18gG5VyKX4rlp5ZQI41x5VpPliVyMDzK8MrgBrbyg7L4o7D QxHCMuIjHmtN3mPACY7rF9LNHffkCd7i0p/OxGt/bpc9vJbcDxXtb95Oyku8N/RwsXdR JgbA== MIME-Version: 1.0 X-Received: by 10.60.92.231 with SMTP id cp7mr9038598oeb.66.1387337864422; Tue, 17 Dec 2013 19:37:44 -0800 (PST) Received: by 10.76.158.225 with HTTP; Tue, 17 Dec 2013 19:37:44 -0800 (PST) In-Reply-To: References: Date: Tue, 17 Dec 2013 22:37:44 -0500 Message-ID: Subject: Re: buf_ring in HEAD is racy From: Ryan Stone To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 Cc: Jack F Vogel , Michael Tuexen , freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2013 03:37:45 -0000 On Tue, Dec 17, 2013 at 11:49 AM, Adrian Chadd wrote: > Try: > > Index: sys/dev/ixgbe/ixgbe.c > =================================================================== > --- sys/dev/ixgbe/ixgbe.c (revision 2995) > +++ sys/dev/ixgbe/ixgbe.c (working copy) > @@ -5178,6 +5178,7 @@ > struct ixgbe_hw *hw = &adapter->hw; > u32 missed_rx = 0, bprc, lxon, lxoff, total; > u64 total_missed_rx = 0; > + u64 odrops = 0; > > adapter->stats.crcerrs += IXGBE_READ_REG(hw, IXGBE_CRCERRS); > adapter->stats.illerrc += IXGBE_READ_REG(hw, IXGBE_ILLERRC); > @@ -5308,6 +5309,11 @@ > adapter->stats.fcoedwtc += IXGBE_READ_REG(hw, IXGBE_FCOEDWTC); > } > > + /* TX drops */ > + for (int i = 0; i < adapter->num_queues; i++) { > + odrops += adapter->tx_rings[i].br->br_drops; > + } > + > /* Fill out the OS statistics structure */ > ifp->if_ipackets = adapter->stats.gprc; > ifp->if_opackets = adapter->stats.gptc; > @@ -5317,6 +5323,9 @@ > ifp->if_omcasts = adapter->stats.mptc; > ifp->if_collisions = 0; > > + /* TX drops are stored in if_snd for now, not the top level counters */ > + ifp->if_snd.ifq_drops = odrops; > + > /* Rx Errors */ > ifp->if_iqdrops = total_missed_rx; > ifp->if_ierrors = adapter->stats.crcerrs + adapter->stats.rlec; > > > -adrian The trick with this is that if ALTQ is enabled then we will use if_snd queue instead of the buf ring, and if that queue drops then ifq_drops will be incremented. With this patch if ALTQ is in use then we will not count any drops (as you will be overwriting the drop count from the queue). From owner-freebsd-net@FreeBSD.ORG Wed Dec 18 05:22:50 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DEF88A8E; Wed, 18 Dec 2013 05:22:50 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5ADF9146F; Wed, 18 Dec 2013 05:22:49 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.7/8.14.7) with ESMTP id rBI5MmF6060568 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 18 Dec 2013 09:22:48 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.7/8.14.7/Submit) id rBI5Mm88060567; Wed, 18 Dec 2013 09:22:48 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Wed, 18 Dec 2013 09:22:48 +0400 From: Gleb Smirnoff To: Mark Martinec Subject: Re: 10-RC unable to build kernel without INET (i.e. IPv6-only) Message-ID: <20131218052248.GC29088@FreeBSD.org> References: <201312172013.20436.Mark.Martinec+freebsd@ijs.si> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201312172013.20436.Mark.Martinec+freebsd@ijs.si> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2013 05:22:50 -0000 Mark, On Tue, Dec 17, 2013 at 08:13:20PM +0100, Mark Martinec wrote: M> Under 9.2 the following could be used to build an IPv6-only kernel: M> M> /sys/amd64/conf/TEST : M> M> include GENERIC M> makeoptions MKMODULESENV+="WITHOUT_INET_SUPPORT=" M> nooptions INET M> M> M> Now with stable/10 the: M> M> make buildkernel KERNCONF=TEST Just fixed that in stable/10. I expect we will merge the patch to release branch as well. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Wed Dec 18 08:44:58 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 154ED8A3; Wed, 18 Dec 2013 08:44:58 +0000 (UTC) Received: from mail-pa0-x235.google.com (mail-pa0-x235.google.com [IPv6:2607:f8b0:400e:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D3649164F; Wed, 18 Dec 2013 08:44:57 +0000 (UTC) Received: by mail-pa0-f53.google.com with SMTP id hz1so5656397pad.40 for ; Wed, 18 Dec 2013 00:44:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=gySQ0+SYJMFd3tA5L0knQj8JoBIafOi8Ec7a9s/qC10=; b=g9elrydPCNi1yDLDt35JEw3p1vAy73OOy4z7VgRS4I5FXwLph/YNGJ4+8KxC0PChPw BjvpdEjS/0qdZcLCftxhP7uKKq9xX4TjpfMJjAMaEPpNy7QNCho++0e07HLC8aKmtYoX DW9+Bf/RB9TzPylbGKVswzfiSVSL7Au4kFFKdkIiEXG1FG7py7u+v93D52fHvEFXi8kW 2bK+fdY7lwbV0RiUc9hFC569A1uQ5VSfNh/nJt/h2ZajBMinQqIt5UF0mwWgPMHvGfZ+ lO1UUHTD4n8cCl44svnO9cNV1XmoAiKQTo36WzKIKvSDWP1n+NaSRdju1t6xhqJ8aShB 1KGQ== X-Received: by 10.66.216.129 with SMTP id oq1mr32168358pac.75.1387356297533; Wed, 18 Dec 2013 00:44:57 -0800 (PST) Received: from pyunyh@gmail.com (lpe4.p59-icn.cdngp.net. [114.111.62.249]) by mx.google.com with ESMTPSA id er3sm39338181pbb.40.2013.12.18.00.44.52 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 18 Dec 2013 00:44:56 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Wed, 18 Dec 2013 17:44:51 +0900 From: Yonghyeon PYUN Date: Wed, 18 Dec 2013 17:44:51 +0900 To: Shteryana Shopova Subject: Re: RTL8111/8168B NIC & FreeBSD Message-ID: <20131218084450.GB1400@michelle.cdnetworks.com> References: <20131216014704.GB3345@michelle.cdnetworks.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: "freebsd-net@FreeBSD.org" , David Rufino X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2013 08:44:58 -0000 On Mon, Dec 16, 2013 at 09:06:14PM +0200, Shteryana Shopova wrote: > Hm, > > It's neither - > > re0: port > 0x2000-0x20ff mem 0xd1404000-0xd1404fff,0xd1400000-0xd1403fff irq 18 > at device 0.1 on pci4 > re0: Using 1 MSI-X message > re0: ASPM disabled > re0: Chip rev. 0x5c800000 > re0: MAC rev. 0x00000000 Your controller is RTL8411B. Try this one instead. http://people.freebsd.org/~yongari/re/re.rxdv_gated.diff > > cheers, > Shteryana > > > On Mon, Dec 16, 2013 at 3:47 AM, Yonghyeon PYUN wrote: > > On Sun, Dec 15, 2013 at 06:46:35AM +0200, Shteryana Shopova wrote: > >> Hi, > >> > >> I hit the same problem with the same chip on FreeBSD 11.0-CURRENT #0 > >> r258178 - the card would receive nothing or (perhaps not-so-)random > >> crap. ifconfig down/up and disabling RX/TX check-summing makes the > >> card work for me as a temporary workaround. > >> > > > > David's controller is RTL8168G. Are you sure you have the same > > controller? If your controller is RTL8168E-VL(Chip rev. > > 0x2c800000 in dmesg), try the patch at the following URL. > > http://people.freebsd.org/~yongari/re/re.8168evl.diff > > > >> re0@pci0:4:0:1: class=0x020000 card=0x07981025 chip=0x816810ec > >> rev=0x14 hdr=0x00 > >> vendor = 'Realtek Semiconductor Co., Ltd.' > >> device = 'RTL8111/8168B PCI Express Gigabit Ethernet controller' > >> class = network > >> subclass = ethernet > >> > >> cheers, > >> Shteryana > >> > >> > >> On Sat, Dec 14, 2013 at 11:31 PM, David Rufino wrote: > >> > Hello, > >> > > >> > I'm having similar problems using an onboard RTL 8168B NIC. I noticed > >> > there were some recent changes to the re0 driver, so I tried -STABLE > >> > (r258961). The device is detected but can't send or receive packets. > >> > Interestingly it doesn't work in Debian wheezy, but does work fine in > >> > Windows 7 with latest drivers. I'm happy to assist in any way. Here's > >> > some relevant debugging information > >> > > >> > pciconf -l -v > >> > > >> > re0@pci0:1:0:0: class=0x020000 card=0x81161019 chip=0x816810ec rev=0x0c hdr=0x00 > >> > vendor = 'Realtek Semiconductor Co., Ltd.' > >> > device = 'RTL8111/8168B PCI Express Gigabit Ethernet controller' > >> > class = network > >> > subclass = ethernet > >> > > >> > re0: port > >> > 0xe000-0xe0ff mem 0xfea00000-0xfea00fff,0xd0800000-0xd0803fff irq 32 > >> > at device 0.0 on pci1 > >> > re0: Using 1 MSI-X message > >> > re0: ASPM disabled > >> > re0: Chip rev. 0x4c000000 > >> > re0: MAC rev. 0x00000000 > >> > miibus0: on re0 > >> > re0: Ethernet address: 74:27:ea:d3:de:5a > >> > > >> > Thanks!, > >> > David > >> > > >> >> > >> >> Hello, > >> >> > >> >> we are trying to install FreeBSD on a computer that uses the NIC mentioned > >> >> above. The NIC is running under linux without problems, which we've tested > >> >> for several days transferring several GB of data. From owner-freebsd-net@FreeBSD.ORG Wed Dec 18 09:56:19 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7FF77684; Wed, 18 Dec 2013 09:56:19 +0000 (UTC) Received: from rush.bluerosetech.com (rush.bluerosetech.com [IPv6:2607:fc50:1000:9b00::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2251B1CED; Wed, 18 Dec 2013 09:56:19 +0000 (UTC) Received: from chombo.houseloki.net (c-71-236-222-167.hsd1.wa.comcast.net [71.236.222.167]) by rush.bluerosetech.com (Postfix) with ESMTPSA id 6ABB311434; Wed, 18 Dec 2013 01:56:17 -0800 (PST) Received: from [IPv6:2601:7:1680:cf0:a89f:a0ee:d554:9461] (unknown [IPv6:2601:7:1680:cf0:a89f:a0ee:d554:9461]) by chombo.houseloki.net (Postfix) with ESMTPSA id 59D1058C; Wed, 18 Dec 2013 01:56:15 -0800 (PST) Message-ID: <52B1713D.5060203@bluerosetech.com> Date: Wed, 18 Dec 2013 01:56:13 -0800 From: Darren Pilgrim User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: CeDeROM , FreeBSD Questions Mailing List , freebsd-net@freebsd.org Subject: Re: bug in dhclient / dhclient.conf / dhcp-options parser? References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2013 09:56:19 -0000 On 11/19/2013 6:09 AM, CeDeROM wrote: > Hello :-) > > I need to send additional options with dhcp request, so I add "send" > option for an interface in /etc/dhclient.conf, however options are > non-standard and they use option value below 100 and they are not sent > in dhcp request. I have noticed that other options with names known to > dhcp-options and decimal numbers above 100 are sent correctly, so > maybe there is an error in the options number parser, or I do > something wrong? > > Here is my example /etc/dhclient.conf configuation: > > interface "em0"{ > send option-060 "vendor"; <-- does not go out in dhcp request > send option-077 "user"; <-- does not go out in dhcp request > send option-123 "blah123"; <-- goes out in dhcp request > send root-path "blahroot"; <-- goes out in dhcp request > } > > Please advise, quick solution appreciated :-) The "option-n" options only work for unknown codes. Option codes 60 and 77 are known as dhcp-class-identifier and user-class, respectively. Both are formatted as ASCII text. From owner-freebsd-net@FreeBSD.ORG Wed Dec 18 10:00:30 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A0FD19C3; Wed, 18 Dec 2013 10:00:30 +0000 (UTC) Received: from mail-qc0-x229.google.com (mail-qc0-x229.google.com [IPv6:2607:f8b0:400d:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4B0791DA7; Wed, 18 Dec 2013 10:00:30 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id r5so5997986qcx.28 for ; Wed, 18 Dec 2013 02:00:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=cXij2z2+Jvbt9oz8GhOHiHLqOUpVF0XvWXuDr+h0T38=; b=uUzPy6lgfD+wWtnMk/Bl4Rz0xb8E4jlUW9g9W6vCm7RZsifPc8L175mWNVo8xDXp5M XBM4QgPlKgus58GJsHugwltJhbyS3yKLp+xE7ycjNx5fTwzaR44cOY8dvtdiFI/nsDLI 8n1IVq88me4IjVZ+/UjRSPrh5XuVlxESgAaOmR+1qIiNPHFUfaXQRudQu36vfL0n6VPO jjuFpHNXKqrZh8ZkLkFw1h/ACJsGS+fLG1YUSLHMkakuSsc6SXhYHvT+ZWMpq3/bvgCD IABzw5A7JxMvpqfCs2KKa3qn8vJtg/5IkRfMefC7hdMWr5EoVTL65XZ5bgnv+/nFQ2Qz fJ8w== MIME-Version: 1.0 X-Received: by 10.224.67.137 with SMTP id r9mr51858563qai.8.1387360829294; Wed, 18 Dec 2013 02:00:29 -0800 (PST) Sender: tomek.cedro@gmail.com Received: by 10.229.151.73 with HTTP; Wed, 18 Dec 2013 02:00:29 -0800 (PST) Received: by 10.229.151.73 with HTTP; Wed, 18 Dec 2013 02:00:29 -0800 (PST) In-Reply-To: <52B1713D.5060203@bluerosetech.com> References: <52B1713D.5060203@bluerosetech.com> Date: Wed, 18 Dec 2013 11:00:29 +0100 X-Google-Sender-Auth: TlyH65wnE5Ne-L3bZhOv5TEwu2s Message-ID: Subject: Re: bug in dhclient / dhclient.conf / dhcp-options parser? From: CeDeROM To: Darren Pilgrim Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-net@freebsd.org, FreeBSD Questions Mailing List X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2013 10:00:30 -0000 Naah, "option-n" should work for all options aside ascii names that vary ascross implementations :-) Best regards, Tomek -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info On Dec 18, 2013 10:56 AM, "Darren Pilgrim" wrote: > On 11/19/2013 6:09 AM, CeDeROM wrote: > >> Hello :-) >> >> I need to send additional options with dhcp request, so I add "send" >> option for an interface in /etc/dhclient.conf, however options are >> non-standard and they use option value below 100 and they are not sent >> in dhcp request. I have noticed that other options with names known to >> dhcp-options and decimal numbers above 100 are sent correctly, so >> maybe there is an error in the options number parser, or I do >> something wrong? >> >> Here is my example /etc/dhclient.conf configuation: >> >> interface "em0"{ >> send option-060 "vendor"; <-- does not go out in dhcp request >> send option-077 "user"; <-- does not go out in dhcp request >> send option-123 "blah123"; <-- goes out in dhcp request >> send root-path "blahroot"; <-- goes out in dhcp request >> } >> >> Please advise, quick solution appreciated :-) >> > > The "option-n" options only work for unknown codes. Option codes 60 and > 77 are known as dhcp-class-identifier and user-class, respectively. Both > are formatted as ASCII text. > From owner-freebsd-net@FreeBSD.ORG Wed Dec 18 11:07:06 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6862E3FC; Wed, 18 Dec 2013 11:07:06 +0000 (UTC) Received: from rush.bluerosetech.com (rush.bluerosetech.com [IPv6:2607:fc50:1000:9b00::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 39F2912A0; Wed, 18 Dec 2013 11:07:06 +0000 (UTC) Received: from chombo.houseloki.net (c-71-236-222-167.hsd1.wa.comcast.net [71.236.222.167]) by rush.bluerosetech.com (Postfix) with ESMTPSA id C2F7211437; Wed, 18 Dec 2013 03:07:04 -0800 (PST) Received: from [IPv6:2601:7:1680:cf0:a89f:a0ee:d554:9461] (unknown [IPv6:2601:7:1680:cf0:a89f:a0ee:d554:9461]) by chombo.houseloki.net (Postfix) with ESMTPSA id 11E1059B; Wed, 18 Dec 2013 03:07:03 -0800 (PST) Message-ID: <52B181D5.5010208@bluerosetech.com> Date: Wed, 18 Dec 2013 03:07:01 -0800 From: Darren Pilgrim User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: CeDeROM Subject: Re: bug in dhclient / dhclient.conf / dhcp-options parser? References: <52B1713D.5060203@bluerosetech.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, FreeBSD Questions Mailing List X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2013 11:07:06 -0000 On 12/18/2013 2:00 AM, CeDeROM wrote: > Naah, "option-n" should work for all options aside ascii names that vary > ascross implementations :-) No, option-n is only defined for codes where there isn't a meaningful name. The dhclient in FreeBSD has meaningful names for codes 60 and 77, so you must use the meaningful names. From owner-freebsd-net@FreeBSD.ORG Wed Dec 18 12:44:28 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 28793B48; Wed, 18 Dec 2013 12:44:28 +0000 (UTC) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) by mx1.freebsd.org (Postfix) with ESMTP id CECD71AC9; Wed, 18 Dec 2013 12:44:27 +0000 (UTC) Received: from amavis-proxy-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3dkws647BZzGMqH; Wed, 18 Dec 2013 13:44:26 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= message-id:content-transfer-encoding:content-type:content-type :mime-version:in-reply-to:references:user-agent:date:date :subject:subject:organization:from:from:received:received :received; s=jakla2; t=1387370664; x=1389962665; bh=JTwSd86BYd8R svN8fxbPw6Jcb9YmNG155hjsWJAGx7A=; b=lAFb3MzCYYb0RG0qUFPv7fkTXOj4 3AooOJmiDTUWhq99S/7IL4BZR9H6HoKHSBJ1vcV+a4MjL+oKzBWD2XUWxXItkJAj zAnhmFJ1uKPCDz8xDXYoqnRbUGPHd+1/qLjKntUO1NCQszCxRrnVCquwf/+ADMsM MFfo3kWWzPBiKwc= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-proxy-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10012) with ESMTP id TYywJPi91ZHh; Wed, 18 Dec 2013 13:44:24 +0100 (CET) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP; Wed, 18 Dec 2013 13:44:24 +0100 (CET) Received: from neli.ijs.si (neli.ijs.si [IPv6:2001:1470:ff80:88:21c:c0ff:feb1:8c91]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mildred.ijs.si (Postfix) with ESMTPSA id BD88073A; Wed, 18 Dec 2013 13:44:24 +0100 (CET) From: Mark Martinec Organization: J. Stefan Institute To: freebsd-current@freebsd.org, freebsd-net@freebsd.org Subject: Re: 10-RC unable to build kernel without INET (i.e. IPv6-only) Date: Wed, 18 Dec 2013 13:44:24 +0100 User-Agent: KMail/1.13.7 (FreeBSD/9.2-STABLE; KDE/4.10.5; amd64; ; ) References: <201312172013.20436.Mark.Martinec+freebsd@ijs.si> <20131218052248.GC29088@FreeBSD.org> In-Reply-To: <20131218052248.GC29088@FreeBSD.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201312181344.24286.Mark.Martinec+freebsd@ijs.si> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2013 12:44:28 -0000 Gleb, > On Tue, Dec 17, 2013 at 08:13:20PM +0100, Mark Martinec wrote: > M> Under 9.2 the following could be used to build an IPv6-only kernel: > M> include GENERIC > M> makeoptions MKMODULESENV+="WITHOUT_INET_SUPPORT=" > M> nooptions INET > M> Now with stable/10 ... fails while building xen support: > > Just fixed that in stable/10. > I expect we will merge the patch to release branch as well. Perfect, builds and works very well now! Thank you for a very quick response! Mark From owner-freebsd-net@FreeBSD.ORG Wed Dec 18 20:57:06 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 20064FDD; Wed, 18 Dec 2013 20:57:06 +0000 (UTC) Received: from mail-qe0-x22b.google.com (mail-qe0-x22b.google.com [IPv6:2607:f8b0:400d:c02::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id BA4C51322; Wed, 18 Dec 2013 20:57:05 +0000 (UTC) Received: by mail-qe0-f43.google.com with SMTP id jy17so200168qeb.30 for ; Wed, 18 Dec 2013 12:57:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=rqNJWHvr9j0PldYFzWfBQuG4rlzDVGQDrXFp44+jBz0=; b=C8PHgoc3FjtbBNKr9wcXRC0iRCdOsovQr3xbcE++3uVfx9h5iWBQySb9+MCb+bucjq oBmvIwIKcBk/KDuPLwKRUs2de3IfpzHv1NyGB9XU+GzGMUMat+TWKMOVCgISW7WmSBO2 yqlxN0EMGXNaSfSycIXiN7eUgHHhxOFh8/zCl6/axILxa8HRs7M+UUaSGt3uJR8igisQ 84EKgCCno21iozNfX+YkIf7rdro4+neMNGX9x9W52MncQ0n7PzpZx3QxuX8xZZgwK2WF efiob5Wi/IYNRRTAtpyFJJf+gtZBnjc8KWuZXAfn6PH3mgM+DZ0cs927R5qLxketxP/I 82qQ== MIME-Version: 1.0 X-Received: by 10.49.131.5 with SMTP id oi5mr57629586qeb.38.1387400224954; Wed, 18 Dec 2013 12:57:04 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Wed, 18 Dec 2013 12:57:04 -0800 (PST) In-Reply-To: References: Date: Wed, 18 Dec 2013 12:57:04 -0800 X-Google-Sender-Auth: 8IHMCh7QM0asQBoxW89fX57asJo Message-ID: Subject: Re: buf_ring in HEAD is racy From: Adrian Chadd To: Ryan Stone Content-Type: text/plain; charset=ISO-8859-1 Cc: Jack F Vogel , Michael Tuexen , freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2013 20:57:06 -0000 On 17 December 2013 19:37, Ryan Stone wrote: > The trick with this is that if ALTQ is enabled then we will use if_snd > queue instead of the buf ring, and if that queue drops then ifq_drops > will be incremented. With this patch if ALTQ is in use then we will > not count any drops (as you will be overwriting the drop count from > the queue). Right. Well, that means it's a problem for at least the chelsio driver as well. Hm, guess we could save the previous value of the sampled drops counters and add the delta to the send queue? Oh it gets more interesting - ip_output() also increments it if the queue structure itself can't hold it. It doesn't even hold any locks when it does that check so it's racy anyway. Aieee... Let me think about it a bit more. Thanks, -adrian From owner-freebsd-net@FreeBSD.ORG Wed Dec 18 21:04:45 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 31B244EC; Wed, 18 Dec 2013 21:04:45 +0000 (UTC) Received: from mail-qa0-x22a.google.com (mail-qa0-x22a.google.com [IPv6:2607:f8b0:400d:c00::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D57D713D1; Wed, 18 Dec 2013 21:04:44 +0000 (UTC) Received: by mail-qa0-f42.google.com with SMTP id k4so4123494qaq.15 for ; Wed, 18 Dec 2013 13:04:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=bl1uflZ59yHSNTnRn6/Ls3ZRyhVhre4pTBYF+Hdongs=; b=Fy8zigV1vBK85tPNofpjAz9DQtfESZspuP5tF3JjP6cljvWWuy6ZzaAhQPW1XPO/ar di1akPLP+27K9A/Cv+C6R9eIM8Gqx/dCxqP7mI9adJw++HTZVQAolUW4zE+Mm3+R/GOY ZS9dfNJZcbHmnSpsEoZLNP59jbzMuHyYDvtoYYwy8aqTspkaUKBS7ZABvYm0KIz4X0fQ gemfkmjjlePv5yMB0CremaDDA4U9iehgG2oC1URh4TYuhyCPcw6QeOdWetQluJqjFKPT z/klZ/SkEqtZa8Y51izpjaCP6SKosA+yx6IFAwwaXdbSH82dS8K69p/inL6HCUe8XtEn JLpQ== MIME-Version: 1.0 X-Received: by 10.224.50.195 with SMTP id a3mr56612825qag.25.1387400684034; Wed, 18 Dec 2013 13:04:44 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Wed, 18 Dec 2013 13:04:43 -0800 (PST) Date: Wed, 18 Dec 2013 13:04:43 -0800 X-Google-Sender-Auth: 6DxBAGLryJOu4cHWkIza0YuPVOk Message-ID: Subject: Removing queue length check in ip_output (was Re: buf_ring in HEAD is racy) From: Adrian Chadd To: Ryan Stone Content-Type: text/plain; charset=ISO-8859-1 Cc: Jack F Vogel , Michael Tuexen , freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2013 21:04:45 -0000 How about we can this check in ip_output(): /* * Verify that we have any chance at all of being able to queue the * packet or packet fragments, unless ALTQ is enabled on the given * interface in which case packetdrop should be done by queueing. */ n = ip_len / mtu + 1; /* how many fragments ? */ if ( #ifdef ALTQ (!ALTQ_IS_ENABLED(&ifp->if_snd)) && #endif /* ALTQ */ (ifp->if_snd.ifq_len + n) >= ifp->if_snd.ifq_maxlen ) { error = ENOBUFS; IPSTAT_INC(ips_odropped); ifp->if_snd.ifq_drops += n; goto bad; } .. it's totally bogus in an if_transmit / SMP world. There's no locking and there's no guarantee that there will be headroom in the queue between this point and the later call to the if_output() method. -adrian From owner-freebsd-net@FreeBSD.ORG Wed Dec 18 22:40:32 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4C627743; Wed, 18 Dec 2013 22:40:32 +0000 (UTC) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 09B711E36; Wed, 18 Dec 2013 22:40:31 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.7/8.14.7) with ESMTP id rBIMeTX3006363; Wed, 18 Dec 2013 17:40:29 -0500 (EST) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.7/8.14.4/Submit) id rBIMeTUE006362; Wed, 18 Dec 2013 17:40:29 -0500 (EST) (envelope-from wollman) Date: Wed, 18 Dec 2013 17:40:29 -0500 (EST) From: Garrett Wollman Message-Id: <201312182240.rBIMeTUE006362@hergotha.csail.mit.edu> To: adrian@freebsd.org Subject: Re: Removing queue length check in ip_output (was Re: buf_ring in HEAD is racy) In-Reply-To: Organization: none X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (hergotha.csail.mit.edu [127.0.0.1]); Wed, 18 Dec 2013 17:40:30 -0500 (EST) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu Cc: freebsd-net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2013 22:40:32 -0000 In article , adrian@freebsd.org writes: >.. it's totally bogus in an if_transmit / SMP world. There's no >locking and there's no guarantee that there will be headroom in the >queue between this point and the later call to the if_output() method. The theory is that when transmitting fragments, it's desirable to have all-or-nothing transmit behavior with respect to the original packet. Transmitting incomplete fragmented packets wastes resources on the local CPU, in the network, and on the remote system, so you should not do so when it is possible to avoid it. Ideally, there would be an if_reserve(ifp, N) interface that hands you a token that you can then pass back N times to if_transmit(). But it's not clear to me how common or necessary this is, except perhaps on very busy authoritative DNS servers. (Are there any other applications for fragmented UDP these days?) So it might not be worth the effort. -GAWollman From owner-freebsd-net@FreeBSD.ORG Wed Dec 18 23:51:46 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E56AD29C for ; Wed, 18 Dec 2013 23:51:46 +0000 (UTC) Received: from mail-qa0-x236.google.com (mail-qa0-x236.google.com [IPv6:2607:f8b0:400d:c00::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A3EAC1383 for ; Wed, 18 Dec 2013 23:51:46 +0000 (UTC) Received: by mail-qa0-f54.google.com with SMTP id f11so970073qae.6 for ; Wed, 18 Dec 2013 15:51:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=EFR7OaUHPHlPbWtvJyrCyrGHKW5BLidHBX7iLSZL0mk=; b=Rs5aU9hkEhBPwCQuvC9Hdut7Jlsj8BlibPlUoZQGx4DvuAJGDifWRJKQ3lyF0DwMwy gPC6DPN36KMyoS+YBREagD5/K9z1fnx8QCSFZ/JygJYbA7HsT9dF+FQWCvLgOkI44vq9 bduYTEIiqP/aQOXNrSnNthBuHU/GGuSADoVRayHNSadaQvv3IV2pl8Cs4/3xEZKh1Gky no5+7090iIa279YUljfcQNvARh1KujVPtMJFnA7LrbUIa9+9dwFufoCMJwRbInOu5rpD ay4/Mt1tJRtj9FcgGY5PkOKjlJOq2+PuFUBPceinZGEXO1gwg40462WbaMopZGIQ6cUg FwKQ== MIME-Version: 1.0 X-Received: by 10.224.50.195 with SMTP id a3mr58003667qag.25.1387410705254; Wed, 18 Dec 2013 15:51:45 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Wed, 18 Dec 2013 15:51:45 -0800 (PST) In-Reply-To: <201312182240.rBIMeTUE006362@hergotha.csail.mit.edu> References: <201312182240.rBIMeTUE006362@hergotha.csail.mit.edu> Date: Wed, 18 Dec 2013 15:51:45 -0800 X-Google-Sender-Auth: j0cOnICd6tDnQbAJ9_vJpDtUyzA Message-ID: Subject: Re: Removing queue length check in ip_output (was Re: buf_ring in HEAD is racy) From: Adrian Chadd To: Garrett Wollman Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2013 23:51:47 -0000 On 18 December 2013 14:40, Garrett Wollman wrote: > In article > , > adrian@freebsd.org writes: > >>.. it's totally bogus in an if_transmit / SMP world. There's no >>locking and there's no guarantee that there will be headroom in the >>queue between this point and the later call to the if_output() method. > > The theory is that when transmitting fragments, it's desirable to have > all-or-nothing transmit behavior with respect to the original packet. > Transmitting incomplete fragmented packets wastes resources on the > local CPU, in the network, and on the remote system, so you should not > do so when it is possible to avoid it. Right, but this API definitely can't guarantee this as it presently stands anyway. > Ideally, there would be an if_reserve(ifp, N) interface that hands you > a token that you can then pass back N times to if_transmit(). But > it's not clear to me how common or necessary this is, except perhaps > on very busy authoritative DNS servers. (Are there any other > applications for fragmented UDP these days?) So it might not be worth > the effort. Ideally we'd pass a list of fragments to the next layer up and such and said layers would treat that list as a "send all or nothing" unit. I have the same issues in the wifi stack with 802.11 fragments. -a From owner-freebsd-net@FreeBSD.ORG Thu Dec 19 08:41:50 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1DC563A3; Thu, 19 Dec 2013 08:41:50 +0000 (UTC) Received: from mail-qa0-x233.google.com (mail-qa0-x233.google.com [IPv6:2607:f8b0:400d:c00::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AE8BD1978; Thu, 19 Dec 2013 08:41:49 +0000 (UTC) Received: by mail-qa0-f51.google.com with SMTP id o15so1274334qap.10 for ; Thu, 19 Dec 2013 00:41:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=gBtv8ODrVt588et2gqlsuc/aLl8M9tef4HgwDXKcvXs=; b=cn3DGRJnugyDHCQbIDGn1mu2XHblgfi3pmi6bXPFrOZ3thLVjIhCm9pMQAMOZwF1p6 FbhX1YKCWQDkpR8gSTZkZ3wmgCS2/+ATDfd592Lnep1gsk6S5e2cqpKXo0mEqxVuyWnK kYOHw/Ju3FSBv06qirMdYgm0CeKYiG6vZahQI2sX25aMibY1JB0OXAUdYAjuPufnwvXH aNu391W9JKPxZqH+1BPw2EhExggMBGRu573gQ96KI2l5kq4x4VRt9QKFywH8Fn/69R+Q CJ465FqgT/mcbbCYIFIuwLAfUQrqid6hy+XNMlDu/KGsxNnUhmwcj7z0f/wOhrFOn3qi AX4w== MIME-Version: 1.0 X-Received: by 10.49.17.232 with SMTP id r8mr426309qed.74.1387442508859; Thu, 19 Dec 2013 00:41:48 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Thu, 19 Dec 2013 00:41:48 -0800 (PST) In-Reply-To: <1387204500.12061.60192349.19EAE1B4@webmail.messagingengine.com> References: <1387204500.12061.60192349.19EAE1B4@webmail.messagingengine.com> Date: Thu, 19 Dec 2013 00:41:48 -0800 X-Google-Sender-Auth: TBvukpmcBvRNrNDuV--ZZIqq5Fg Message-ID: Subject: Re: 10.0-RC1: bad mbuf leak? From: Adrian Chadd To: Mark Felder Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net , FreeBSD Stable Mailing List X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2013 08:41:50 -0000 Hm, try reverting just the em code to that from a 10.0-BETA? Just in case something changed there? -a On 16 December 2013 06:35, Mark Felder wrote: > Hi all, > > I think I'm experiencing a bad mbuf leak or something of the sort and I > don't know how to diagnose this further. > > I have a machine at home that is mostly used for transcoding video for > viewing on my TV via the multimedia/plexmediaserver port. This software > runs in a jail and gets the actual files from my NAS via NFSv4. It's a > pretty simple setup and sits idle unless I am watching TV. > > Between the 10.0-BETAs and the 10.0-RC1 did something network related > that could affect mbufs change? Ever since I upgraded this machine to > RC1 it has been "crashing", which I diagnosed as actually being an mbuf > exhaustion. Raising the mbufs brings it back to life, and it does > mention the exhaustion on the system console. > > Last night, for example, I rebooted the machine and it has been sitting > mostly idle. I wake up this morning to see this: > > # vmstat -z > > ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP > mbuf_packet: 256, 6511095, 1023, 1727, 8322474, 0, > 0 > mbuf: 256, 6511095, 2811247, 1563,56000603,121933, > 0 > mbuf_cluster: 2048, 1017358, 2750, 0, 2750,2740, > 0 > mbuf_jumbo_page: 4096, 508679, 0, 152, 2831466, 137, 0 > > # netstat -m > 812270/3290/2815560 mbufs in use (current/cache/total) > 1023/1727/2750/1017358 mbuf clusters in use (current/cache/total/max) > 1023/1727 mbuf+clusters out of packet secondary zone in use > (current/cache) > 0/152/152/508679 4k (page size) jumbo clusters in use > (current/cache/total/max) > 0/0/0/150719 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/84779 16k jumbo clusters in use (current/cache/total/max) > 705113K/4884K/709998K bytes allocated to network (current/cache/total) > 121933/2740/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) > 137/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 0 requests for I/O initiated by sendfile > > > The network interface is em(4). > > Things I've tried: > > - restarting all software/services including the jail > - down/up the network interface > > The only thing that works is rebooting. > > Also, the only possible "strange" part of this setup is that the NFS > mounts used by the jail are not direct. They're actually nullfs mounted > into the jail as I want access to them outside of the jail as well. Not > sure if nullfs+nfs could do something this bizarre. > > If anyone has any hints on what I can do to track this down it would be > appreciated. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Thu Dec 19 09:09:49 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DCEBA8C1; Thu, 19 Dec 2013 09:09:48 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0FB921B46; Thu, 19 Dec 2013 09:09:47 +0000 (UTC) Received: from [10.225.9.5] (unknown [194.95.73.101]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 333431C0C0692; Thu, 19 Dec 2013 10:09:44 +0100 (CET) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) Subject: Re: 10.0-RC1: bad mbuf leak? From: Michael Tuexen In-Reply-To: Date: Thu, 19 Dec 2013 10:09:45 +0100 Content-Transfer-Encoding: 7bit Message-Id: References: <1387204500.12061.60192349.19EAE1B4@webmail.messagingengine.com> To: Adrian Chadd X-Mailer: Apple Mail (2.1510) Cc: FreeBSD Net , FreeBSD Stable Mailing List , Mark Felder X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2013 09:09:49 -0000 On Dec 19, 2013, at 9:41 AM, Adrian Chadd wrote: > Hm, try reverting just the em code to that from a 10.0-BETA? Just in > case something changed there? I saw a similar behaviour without the patches we are discussing (regarding ignoring the error). Best regards Michael > > > > -a > > On 16 December 2013 06:35, Mark Felder wrote: >> Hi all, >> >> I think I'm experiencing a bad mbuf leak or something of the sort and I >> don't know how to diagnose this further. >> >> I have a machine at home that is mostly used for transcoding video for >> viewing on my TV via the multimedia/plexmediaserver port. This software >> runs in a jail and gets the actual files from my NAS via NFSv4. It's a >> pretty simple setup and sits idle unless I am watching TV. >> >> Between the 10.0-BETAs and the 10.0-RC1 did something network related >> that could affect mbufs change? Ever since I upgraded this machine to >> RC1 it has been "crashing", which I diagnosed as actually being an mbuf >> exhaustion. Raising the mbufs brings it back to life, and it does >> mention the exhaustion on the system console. >> >> Last night, for example, I rebooted the machine and it has been sitting >> mostly idle. I wake up this morning to see this: >> >> # vmstat -z >> >> ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP >> mbuf_packet: 256, 6511095, 1023, 1727, 8322474, 0, >> 0 >> mbuf: 256, 6511095, 2811247, 1563,56000603,121933, >> 0 >> mbuf_cluster: 2048, 1017358, 2750, 0, 2750,2740, >> 0 >> mbuf_jumbo_page: 4096, 508679, 0, 152, 2831466, 137, 0 >> >> # netstat -m >> 812270/3290/2815560 mbufs in use (current/cache/total) >> 1023/1727/2750/1017358 mbuf clusters in use (current/cache/total/max) >> 1023/1727 mbuf+clusters out of packet secondary zone in use >> (current/cache) >> 0/152/152/508679 4k (page size) jumbo clusters in use >> (current/cache/total/max) >> 0/0/0/150719 9k jumbo clusters in use (current/cache/total/max) >> 0/0/0/84779 16k jumbo clusters in use (current/cache/total/max) >> 705113K/4884K/709998K bytes allocated to network (current/cache/total) >> 121933/2740/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) >> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) >> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) >> 137/0/0 requests for jumbo clusters denied (4k/9k/16k) >> 0 requests for sfbufs denied >> 0 requests for sfbufs delayed >> 0 requests for I/O initiated by sendfile >> >> >> The network interface is em(4). >> >> Things I've tried: >> >> - restarting all software/services including the jail >> - down/up the network interface >> >> The only thing that works is rebooting. >> >> Also, the only possible "strange" part of this setup is that the NFS >> mounts used by the jail are not direct. They're actually nullfs mounted >> into the jail as I want access to them outside of the jail as well. Not >> sure if nullfs+nfs could do something this bizarre. >> >> If anyone has any hints on what I can do to track this down it would be >> appreciated. >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-net@FreeBSD.ORG Thu Dec 19 15:16:09 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 24A9E868; Thu, 19 Dec 2013 15:16:09 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9465D1958; Thu, 19 Dec 2013 15:16:08 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.7/8.14.7) with ESMTP id rBJFG6Oo071117 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 19 Dec 2013 19:16:06 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.7/8.14.7/Submit) id rBJFG6OK071116; Thu, 19 Dec 2013 19:16:06 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Thu, 19 Dec 2013 19:16:06 +0400 From: Gleb Smirnoff To: Adrian Chadd Subject: Re: Removing queue length check in ip_output (was Re: buf_ring in HEAD is racy) Message-ID: <20131219151606.GB71033@FreeBSD.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.22 (2013-10-16) Cc: Jack F Vogel , Michael Tuexen , Ryan Stone , freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2013 15:16:09 -0000 On Wed, Dec 18, 2013 at 01:04:43PM -0800, Adrian Chadd wrote: A> How about we can this check in ip_output(): A> A> /* A> * Verify that we have any chance at all of being able to queue the A> * packet or packet fragments, unless ALTQ is enabled on the given A> * interface in which case packetdrop should be done by queueing. A> */ A> n = ip_len / mtu + 1; /* how many fragments ? */ A> if ( A> #ifdef ALTQ A> (!ALTQ_IS_ENABLED(&ifp->if_snd)) && A> #endif /* ALTQ */ A> (ifp->if_snd.ifq_len + n) >= ifp->if_snd.ifq_maxlen ) { A> error = ENOBUFS; A> IPSTAT_INC(ips_odropped); A> ifp->if_snd.ifq_drops += n; A> goto bad; A> } A> A> .. it's totally bogus in an if_transmit / SMP world. There's no A> locking and there's no guarantee that there will be headroom in the A> queue between this point and the later call to the if_output() method. My vote is to remove this. But better leave a comment that we need an API to send a bunch of packets (or fail them all) via an interface. -- Totus tuus, Glebius. From owner-freebsd-net@FreeBSD.ORG Thu Dec 19 17:11:08 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C5D54901; Thu, 19 Dec 2013 17:11:08 +0000 (UTC) Received: from mail-qa0-x22a.google.com (mail-qa0-x22a.google.com [IPv6:2607:f8b0:400d:c00::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 601061452; Thu, 19 Dec 2013 17:11:08 +0000 (UTC) Received: by mail-qa0-f42.google.com with SMTP id k4so5001487qaq.15 for ; Thu, 19 Dec 2013 09:11:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=sJ/YtMJKqvV4Ij1Zp38DmuQatLkgGEh9AIl/Vh17d+Q=; b=GZkePLfXOEunfai6LZPSE2lKf6ARRKssjlzjeTJh/ZVq0mm9NNxky7s4tWbtj4EWFf EG6HH2naNOosJTpgc0DiS7htWN8LIMB63d/fG2bXr+VgPLl+E3Z6UQoRGecq6Y4D5ZMN HetxF7uut0qax0LT25fZSAIiTrSfIgYWUYft4WO24Y+t/396UvG4m4qjkhUFnoeTUSS1 //i0TLSnycvguQQiOWxqK9oIFfrUJxI//uUCYeodaXPSe07fNoo1lh/ipo2W3uCw70uF 8P4vu9of8QmxRLJjCvH6yC8KFwJt00x4bKsADkp0busGPYKxc6Kqv4aXGg8VQSC27+56 pKhw== MIME-Version: 1.0 X-Received: by 10.224.16.204 with SMTP id p12mr5088173qaa.26.1387472962850; Thu, 19 Dec 2013 09:09:22 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Thu, 19 Dec 2013 09:09:22 -0800 (PST) In-Reply-To: <20131219151606.GB71033@FreeBSD.org> References: <20131219151606.GB71033@FreeBSD.org> Date: Thu, 19 Dec 2013 09:09:22 -0800 X-Google-Sender-Auth: -8hrE1D_xdZLhzdWQZNLYO2LaU4 Message-ID: Subject: Re: Removing queue length check in ip_output (was Re: buf_ring in HEAD is racy) From: Adrian Chadd To: Gleb Smirnoff Content-Type: text/plain; charset=ISO-8859-1 Cc: Jack F Vogel , Michael Tuexen , Ryan Stone , freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2013 17:11:08 -0000 On 19 December 2013 07:16, Gleb Smirnoff wrote: > On Wed, Dec 18, 2013 at 01:04:43PM -0800, Adrian Chadd wrote: > A> How about we can this check in ip_output(): > A> > A> /* > A> * Verify that we have any chance at all of being able to queue the > A> * packet or packet fragments, unless ALTQ is enabled on the given > A> * interface in which case packetdrop should be done by queueing. > A> */ > A> n = ip_len / mtu + 1; /* how many fragments ? */ > A> if ( > A> #ifdef ALTQ > A> (!ALTQ_IS_ENABLED(&ifp->if_snd)) && > A> #endif /* ALTQ */ > A> (ifp->if_snd.ifq_len + n) >= ifp->if_snd.ifq_maxlen ) { > A> error = ENOBUFS; > A> IPSTAT_INC(ips_odropped); > A> ifp->if_snd.ifq_drops += n; > A> goto bad; > A> } > A> > A> .. it's totally bogus in an if_transmit / SMP world. There's no > A> locking and there's no guarantee that there will be headroom in the > A> queue between this point and the later call to the if_output() method. > > My vote is to remove this. But better leave a comment that we need > an API to send a bunch of packets (or fail them all) via an interface. Yes. I think we need some path to evolve if_transmit to a) transmit multiple frames as a transaction (eg a set of IP or 802.11 fragments) at once, and b) be able to transmit multiple transactions at once. Right now chaining packets together via m_nextpkt can only be used for one or the other. We'd have to set flags in the mbufs to delineate the transaction boundaries and .. ew. -a From owner-freebsd-net@FreeBSD.ORG Thu Dec 19 19:59:07 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AB6ADB6F; Thu, 19 Dec 2013 19:59:07 +0000 (UTC) Received: from mail-qa0-x22c.google.com (mail-qa0-x22c.google.com [IPv6:2607:f8b0:400d:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 41A4C1327; Thu, 19 Dec 2013 19:59:07 +0000 (UTC) Received: by mail-qa0-f44.google.com with SMTP id i13so5212067qae.10 for ; Thu, 19 Dec 2013 11:59:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=OV33OWgXJAkYasNGb2tgucrVkNOYSSB686kbGz+G8Fk=; b=WPDnfA46xLYPTKCF0TYDdSnPjTO1Kpares3qMxvQkceVBC+LRckAGiXodUcks2fM79 9abetrp7v2KqriZ+ccu72aJFCpNtzgve0gk6HWQUfFQzvCCCufFF0TqOFICEditeGEIp r/XneX3vuwVx4+Ms9WeKDHsSjQ/bQxU92VY5Wsw7R0X4ILNlA3ADn2XRXiHM16MAovxU +kjUm+VyMmDmfdcB1Ka6HzGUJKuYXH7fvYT94d/hpDHcLy9/IGc6DlZ58q9mWeTrWi5y U0i28tDIOxQO9eYjlu2Vx/Bmqzaw0F+5KLjEheIv4IYc7oOMCLhOmI2tAlzWNplb7cKz 9DGQ== MIME-Version: 1.0 X-Received: by 10.49.129.38 with SMTP id nt6mr6564570qeb.78.1387483146476; Thu, 19 Dec 2013 11:59:06 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Thu, 19 Dec 2013 11:59:06 -0800 (PST) In-Reply-To: References: <20131219151606.GB71033@FreeBSD.org> Date: Thu, 19 Dec 2013 11:59:06 -0800 X-Google-Sender-Auth: 8d637BCPJ7zKxakOVZeTjrzO66Y Message-ID: Subject: Re: Removing queue length check in ip_output (was Re: buf_ring in HEAD is racy) From: Adrian Chadd To: Gleb Smirnoff Content-Type: text/plain; charset=ISO-8859-1 Cc: Jack F Vogel , Michael Tuexen , Ryan Stone , freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2013 19:59:07 -0000 How's this? Index: sys/netinet/ip_output.c =================================================================== --- sys/netinet/ip_output.c (revision 259474) +++ sys/netinet/ip_output.c (working copy) @@ -123,7 +123,9 @@ struct mbuf *m0; int hlen = sizeof (struct ip); int mtu; +#if 0 int n; /* scratchpad */ +#endif int error = 0; struct sockaddr_in *dst; const struct sockaddr_in *gw; @@ -431,6 +433,25 @@ } /* + * Both in the SMP world, pre-emption world if_transmit() world, + * the following code doesn't really function as intended any further. + * + * + There can and will be multiple CPUs running this code path + * in parallel, and we do no lock holding when checking the + * queue depth; + * + And since other threads can be running concurrently, even if + * we do pass this check, another thread may queue some frames + * before this thread does and it will end up partially or fully + * failing to send anyway; + * + if_transmit() based drivers don't necessarily set ifq_len + * at all. + * + * This should be replaced with a method of pushing an entire list + * of fragment frames to the driver and have the driver decide + * whether it can queue or not queue the entire set. + */ +#if 0 + /* * Verify that we have any chance at all of being able to queue the * packet or packet fragments, unless ALTQ is enabled on the given * interface in which case packetdrop should be done by queueing. @@ -446,6 +467,7 @@ ifp->if_snd.ifq_drops += n; goto bad; } +#endif /* * Look for broadcast address and -a From owner-freebsd-net@FreeBSD.ORG Thu Dec 19 20:02:28 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A0E3DD5D; Thu, 19 Dec 2013 20:02:28 +0000 (UTC) Received: from mail-qc0-x236.google.com (mail-qc0-x236.google.com [IPv6:2607:f8b0:400d:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3B82A13A5; Thu, 19 Dec 2013 20:02:28 +0000 (UTC) Received: by mail-qc0-f182.google.com with SMTP id e16so1415582qcx.13 for ; Thu, 19 Dec 2013 12:02:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=qJMm4nownejKCUhRZ8EPTQ5pfhPt6bh+h2BiexpNTxg=; b=FyQtD0V4vV194zQN54pcVE4MmXMWSjyLfyy3z+zSqZK82uMjFcVcLGJnkMTDvN/VGY wzUtorzIK8mS0A1SulrQ0W5Nd/i8FUPkiLeda0v57s803blfh1lDhvfINcuv6Hr9dLn+ 00maD2lI+ntzm73Ser5X7evvZ5m3BTyN5b1T+TO/d1cIqmQ4CkK2EeWonHwN6lH4RTTJ QrdgAnRC6MJonByUEEkim4GSIaDeike1EJhxrZSDf2UsPMQAYNkDyEsb87OEFf4TWgNp 59gYyZSyRjvBdxugm+fDAh4afHSZgxKCO5oAJTzzWqASL+DCtoaIInkVC8C+06N2ogia 3ZJQ== MIME-Version: 1.0 X-Received: by 10.229.24.4 with SMTP id t4mr6601693qcb.13.1387483347405; Thu, 19 Dec 2013 12:02:27 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Thu, 19 Dec 2013 12:02:27 -0800 (PST) In-Reply-To: References: <1387204500.12061.60192349.19EAE1B4@webmail.messagingengine.com> Date: Thu, 19 Dec 2013 12:02:27 -0800 X-Google-Sender-Auth: iITaZI1yTLhSSSA2RjkuQshU2ig Message-ID: Subject: Re: 10.0-RC1: bad mbuf leak? From: Adrian Chadd To: Michael Tuexen Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net , FreeBSD Stable Mailing List , Mark Felder X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2013 20:02:28 -0000 On 19 December 2013 01:09, Michael Tuexen wrote: > On Dec 19, 2013, at 9:41 AM, Adrian Chadd wrote: > >> Hm, try reverting just the em code to that from a 10.0-BETA? Just in >> case something changed there? > I saw a similar behaviour without the patches we are discussing (regarding > ignoring the error). Hm. Why was that? -a From owner-freebsd-net@FreeBSD.ORG Thu Dec 19 20:03:35 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2EA611F1; Thu, 19 Dec 2013 20:03:35 +0000 (UTC) Received: from mail-pb0-x229.google.com (mail-pb0-x229.google.com [IPv6:2607:f8b0:400e:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E0CE613D0; Thu, 19 Dec 2013 20:03:34 +0000 (UTC) Received: by mail-pb0-f41.google.com with SMTP id jt11so1621464pbb.0 for ; Thu, 19 Dec 2013 12:03:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=51dWnUG7SFfG/wT0hLsbTLAA3fqf9gqN6R9veaqdoE8=; b=a7wIXYMWK3HVGsAF+Iq092s1nQmquPhvNyOFRKgMA9H2popLxo6rWD9rRwi6v3jnC4 GlRX3MmJXyZHeaE3L7rYDY+V9jw7Xi6XvpkvBd53+iIgjQ0rAexAK2qmCs2FSOLylxbz 9ZSC61tikW6JG71ykzQ4MYTxydc3ErJlyk0i4y4/bx/tcvr/JpOx3tmlKnOJVmEq9PNg TvMiwbc44bBNe+uup/E441l/vmSAfdG0VIiQ3llaqFUP4R9jxrFgLeHcAo1hn4HisJ5z lxJXTDQpDNIfKsBBwcYtvRPQIAVrx2WTB9dNzbu4ze8C1XYlwpHBwg44x8t+IAdDxMv7 /NQA== X-Received: by 10.69.21.1 with SMTP id hg1mr3751254pbd.67.1387483414124; Thu, 19 Dec 2013 12:03:34 -0800 (PST) Received: from [10.192.166.0] (stargate.chelsio.com. [67.207.112.58]) by mx.google.com with ESMTPSA id ye1sm11908815pab.19.2013.12.19.12.03.31 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 19 Dec 2013 12:03:32 -0800 (PST) Sender: Navdeep Parhar Message-ID: <52B35113.9050108@FreeBSD.org> Date: Thu, 19 Dec 2013 12:03:31 -0800 From: Navdeep Parhar User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: Adrian Chadd , Gleb Smirnoff Subject: Re: Removing queue length check in ip_output (was Re: buf_ring in HEAD is racy) References: <20131219151606.GB71033@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Jack F Vogel , Michael Tuexen , Ryan Stone , freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2013 20:03:35 -0000 On 12/19/13 11:59, Adrian Chadd wrote: > How's this? > > Index: sys/netinet/ip_output.c > =================================================================== > --- sys/netinet/ip_output.c (revision 259474) > +++ sys/netinet/ip_output.c (working copy) > @@ -123,7 +123,9 @@ > struct mbuf *m0; > int hlen = sizeof (struct ip); > int mtu; > +#if 0 > int n; /* scratchpad */ > +#endif > int error = 0; > struct sockaddr_in *dst; > const struct sockaddr_in *gw; > @@ -431,6 +433,25 @@ > } > > /* > + * Both in the SMP world, pre-emption world if_transmit() world, > + * the following code doesn't really function as intended any further. Doesn't make sense in the tx multiqueue world either. (which queue is this piece of code checking?) Navdeep From owner-freebsd-net@FreeBSD.ORG Thu Dec 19 23:40:51 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BDD48E28; Thu, 19 Dec 2013 23:40:51 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8EDA614A2; Thu, 19 Dec 2013 23:40:51 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id rBJNenIS049531 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 19 Dec 2013 15:40:49 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id rBJNemHi049530; Thu, 19 Dec 2013 15:40:48 -0800 (PST) (envelope-from jmg) Date: Thu, 19 Dec 2013 15:40:48 -0800 From: John-Mark Gurney To: Adrian Chadd Subject: Re: Removing queue length check in ip_output (was Re: buf_ring in HEAD is racy) Message-ID: <20131219234048.GV99167@funkthat.com> Mail-Followup-To: Adrian Chadd , Gleb Smirnoff , Jack F Vogel , Michael Tuexen , Ryan Stone , freebsd-net References: <20131219151606.GB71033@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Thu, 19 Dec 2013 15:40:49 -0800 (PST) Cc: Jack F Vogel , Michael Tuexen , Ryan Stone , freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2013 23:40:51 -0000 Adrian Chadd wrote this message on Thu, Dec 19, 2013 at 11:59 -0800: > + * + There can and will be multiple CPUs running this code path > + * in parallel, and we do no lock holding when checking the > + * queue depth; I think you mean we do no hold the lock when checking... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-net@FreeBSD.ORG Thu Dec 19 23:54:35 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B20E6178; Thu, 19 Dec 2013 23:54:35 +0000 (UTC) Received: from mail-qe0-x236.google.com (mail-qe0-x236.google.com [IPv6:2607:f8b0:400d:c02::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 335FA159D; Thu, 19 Dec 2013 23:54:35 +0000 (UTC) Received: by mail-qe0-f54.google.com with SMTP id cy11so1782506qeb.27 for ; Thu, 19 Dec 2013 15:54:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=pdI5f6YWWF2AjvhQVhDM+lxcvs9VAg0uD7rJ+bDs6SU=; b=s99UeqL6m7k6xJSl2lGvRNsxA/BofpW7uoI3eyqMEdvEkE7iHdsB7JiBHnHRUAsgDY 8Gozw+Rq7k3IszO7au/kZpbpL2RiAYTjkK/A0tiKd9NzR98nXeXctBUPp5mpHrwdgdDG TDNJYqO3pWPARazP+oKeC1dDx3NCA8gYjJsqezLOpcItqvqOqwMezJ0PR54yo6I5oZnV os6h3mVkHiZe+AAc1ip1623SKmw2XY/HmOw7GyE2DUFR2hrBT/DBHewdb/1vh7p0Vd+G slQZiyvUkD0Pms0SlccByknz4qMH2Rhx8AppCrAbmWoXbcxQymKMtefnE8PO7WiMCJbF B3TQ== MIME-Version: 1.0 X-Received: by 10.49.24.211 with SMTP id w19mr8564468qef.9.1387497272919; Thu, 19 Dec 2013 15:54:32 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.53.200 with HTTP; Thu, 19 Dec 2013 15:54:32 -0800 (PST) Received: by 10.224.53.200 with HTTP; Thu, 19 Dec 2013 15:54:32 -0800 (PST) In-Reply-To: <52B35113.9050108@FreeBSD.org> References: <20131219151606.GB71033@FreeBSD.org> <52B35113.9050108@FreeBSD.org> Date: Thu, 19 Dec 2013 15:54:32 -0800 X-Google-Sender-Auth: y5g_uqi32ShvxkiRF_fnFiJml8I Message-ID: Subject: Re: Removing queue length check in ip_output (was Re: buf_ring in HEAD is racy) From: Adrian Chadd To: Navdeep Parhar Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Jack F Vogel , Michael Tuexen , Ryan Stone , freebsd-net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2013 23:54:35 -0000 Its checking the ifsnd queue. That just doesn't do anything sane or consistent with drivers that implement if transmit let alone multiqueue drivers. Adrian On Dec 19, 2013 2:03 PM, "Navdeep Parhar" wrote: > On 12/19/13 11:59, Adrian Chadd wrote: > > How's this? > > > > Index: sys/netinet/ip_output.c > > =================================================================== > > --- sys/netinet/ip_output.c (revision 259474) > > +++ sys/netinet/ip_output.c (working copy) > > @@ -123,7 +123,9 @@ > > struct mbuf *m0; > > int hlen = sizeof (struct ip); > > int mtu; > > +#if 0 > > int n; /* scratchpad */ > > +#endif > > int error = 0; > > struct sockaddr_in *dst; > > const struct sockaddr_in *gw; > > @@ -431,6 +433,25 @@ > > } > > > > /* > > + * Both in the SMP world, pre-emption world if_transmit() world, > > + * the following code doesn't really function as intended any > further. > > Doesn't make sense in the tx multiqueue world either. (which queue is > this piece of code checking?) > > Navdeep > From owner-freebsd-net@FreeBSD.ORG Fri Dec 20 11:57:34 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B73CE9C8; Fri, 20 Dec 2013 11:57:34 +0000 (UTC) Received: from mail-n.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4A2CD1CE0; Fri, 20 Dec 2013 11:57:34 +0000 (UTC) Received: from [192.168.1.200] (p508F063C.dip0.t-ipconnect.de [80.143.6.60]) (Authenticated sender: macmic) by mail-n.franken.de (Postfix) with ESMTP id 356F41C0C0693; Fri, 20 Dec 2013 12:57:31 +0100 (CET) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) Subject: Re: 10.0-RC1: bad mbuf leak? From: Michael Tuexen In-Reply-To: Date: Fri, 20 Dec 2013 12:57:29 +0100 Content-Transfer-Encoding: 7bit Message-Id: References: <1387204500.12061.60192349.19EAE1B4@webmail.messagingengine.com> To: Adrian Chadd X-Mailer: Apple Mail (2.1510) Cc: FreeBSD Net , FreeBSD Stable Mailing List , Mark Felder X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Dec 2013 11:57:34 -0000 On Dec 19, 2013, at 9:02 PM, Adrian Chadd wrote: > On 19 December 2013 01:09, Michael Tuexen > wrote: >> On Dec 19, 2013, at 9:41 AM, Adrian Chadd wrote: >> >>> Hm, try reverting just the em code to that from a 10.0-BETA? Just in >>> case something changed there? >> I saw a similar behaviour without the patches we are discussing (regarding >> ignoring the error). > > Hm. Why was that? I don't know. I have some igb devices nailed down to 100MBit, but I haven't tested. Since this problem also triggered a problem in SCTP, I looked at that first and came up with a patch for the drivers to have correctly in case of drops. However, with my fixes, the mbuf problems went away... I can revert my patches and try to figure you what is going on... The mbufs denied counter was already large after a reboot of the system, without any traffic (SCTP or anything else). Best regards Michael > > > -a > From owner-freebsd-net@FreeBSD.ORG Sat Dec 21 03:27:23 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5F54DF62; Sat, 21 Dec 2013 03:27:23 +0000 (UTC) Received: from exprod6og121.obsmtp.com (exprod6og121.obsmtp.com [64.18.1.237]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6363111DA; Sat, 21 Dec 2013 03:27:21 +0000 (UTC) Received: from peregrine.verisign.com ([216.168.239.74]) (using TLSv1) by exprod6ob121.postini.com ([64.18.5.12]) with SMTP ID DSNKUrUKmdq1IjqB8N9ZP+ZFSZX+Jr8MsHCp@postini.com; Fri, 20 Dec 2013 19:27:22 PST Received: from brn1wnexcas02.vcorp.ad.vrsn.com (brn1wnexcas02.vcorp.ad.vrsn.com [10.173.152.206]) by peregrine.verisign.com (8.13.6/8.13.4) with ESMTP id rBKJJrQP006492 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 20 Dec 2013 14:19:57 -0500 Received: from BRN1WNEXMBX01.vcorp.ad.vrsn.com ([::1]) by brn1wnexcas02.vcorp.ad.vrsn.com ([::1]) with mapi id 14.02.0342.003; Fri, 20 Dec 2013 14:19:52 -0500 From: "Bentkofsky, Michael" To: "freebsd-net@freebsd.org" Subject: =?Windows-1252?Q?kern/185043:_Kernel_panic:_Sleeping_thread_(tid_=85, _pid?= =?Windows-1252?Q?_=85)_owns_a_non-sleepable_lock_from_netinet/in=5Fmulti.?= =?Windows-1252?Q?c?= Thread-Topic: =?Windows-1252?Q?kern/185043:_Kernel_panic:_Sleeping_thread_(tid_=85, _pid?= =?Windows-1252?Q?_=85)_owns_a_non-sleepable_lock_from_netinet/in=5Fmulti.?= =?Windows-1252?Q?c?= Thread-Index: Ac79toNrVM2JIxXOTZiLsJJKOFKxFw== Date: Fri, 20 Dec 2013 19:19:51 +0000 Message-ID: <080FBD5B7A09F845842100A6DE79623346E3C522@BRN1WNEXMBX01.vcorp.ad.vrsn.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.173.152.4] MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: "rwatson@freebsd.org" , "George Neville-Neil \(gnn@freebsd.org\)" , "bms@FreeBSD.org" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Dec 2013 03:27:23 -0000 I have just submitted PR kern/185043 and wanted to follow-up to freebsd-net= with a patch that fixes the obvious paths leaving in_multi_lock incorrectl= y locked. Index: sys/netinet/in_mcast.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/netinet/in_mcast.c (revision 259264) +++ sys/netinet/in_mcast.c (working copy) @@ -1492,7 +1492,7 @@ error =3D inm_merge(inm, imf); if (error) { CTR1(KTR_IGMPV3, "%s: failed to merge inm state", __func__); - goto out_imf_rollback; + goto out_in_multi_locked; } CTR1(KTR_IGMPV3, "%s: doing igmp downcall", __func__); @@ -1500,6 +1500,8 @@ if (error) CTR1(KTR_IGMPV3, "%s: failed igmp downcall", __func__); +out_in_multi_locked: + IN_MULTI_UNLOCK(); out_imf_rollback: @@ -2168,8 +2170,12 @@ if (is_new) { error =3D in_joingroup_locked(ifp, &gsa->sin.sin_addr, imf, &inm); - if (error) + if (error) { + CTR1(KTR_IGMPV3, "%s: in_joingroup_locked failed", + __func__); + IN_MULTI_UNLOCK(); goto out_imo_free; + } imo->imo_membership[idx] =3D inm; } else { CTR1(KTR_IGMPV3, "%s: merge inm state", __func__); @@ -2177,20 +2183,21 @@ if (error) { CTR1(KTR_IGMPV3, "%s: failed to merge inm state", __func__); - goto out_imf_rollback; + goto out_in_multi_locked; } CTR1(KTR_IGMPV3, "%s: doing igmp downcall", __func__); error =3D igmp_change_state(inm); if (error) { CTR1(KTR_IGMPV3, "%s: failed igmp downcall", __func__); - goto out_imf_rollback; + goto out_in_multi_locked; } } +out_in_multi_locked: + IN_MULTI_UNLOCK(); -out_imf_rollback: INP_WLOCK_ASSERT(inp); if (error) { imf_rollback(imf); @@ -2394,7 +2401,7 @@ if (error) { CTR1(KTR_IGMPV3, "%s: failed to merge inm state", __func__); - goto out_imf_rollback; + goto out_in_multi_locked; } CTR1(KTR_IGMPV3, "%s: doing igmp downcall", __func__); @@ -2405,9 +2412,10 @@ } } +out_in_multi_locked: + IN_MULTI_UNLOCK(); -out_imf_rollback: if (error) imf_rollback(imf); else @@ -2641,7 +2649,7 @@ error =3D inm_merge(inm, imf); if (error) { CTR1(KTR_IGMPV3, "%s: failed to merge inm state", __func__); - goto out_imf_rollback; + goto out_in_multi_locked; } CTR1(KTR_IGMPV3, "%s: doing igmp downcall", __func__); @@ -2649,6 +2657,8 @@ if (error) CTR1(KTR_IGMPV3, "%s: failed igmp downcall", __func__); +out_in_multi_locked: + IN_MULTI_UNLOCK(); out_imf_rollback: =3D=3D=3D=3D=3D End of patch =3D=3D=3D=3D=3D The PR also cites a follow-up problem that this patch does not address, nam= ely that the same technique for causing the problem (subscribing to a multi= cast address using quagga=92s ospfd and doing /etc/rc.d/netif restart) can = result in a race condition leading to this kernel assertion: if_freemulti: = protospec not NULL. The PR describes the particular execution path and comm= ents suggesting this might be a known problem. Since I have triggered the c= ase, I am interested in patching this as well and need more time to conside= r a more thorough solution. Input is appreciated, of course. Hopefully the patch above is obvious enough to be accepted. =93This message (including any attachments) is intended only for the use of= the individual or entity to which it is addressed, and may contain informa= tion that is non-public, proprietary, privileged, confidential and exempt f= rom disclosure under applicable law or may be constituted as attorney work = product. If you are not the intended recipient, you are hereby notified tha= t any use, dissemination, distribution, or copying of this communication is= strictly prohibited. If you have received this message in error, notify se= nder immediately and delete this message immediately.=94 From owner-freebsd-net@FreeBSD.ORG Sat Dec 21 19:16:00 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 938965D3; Sat, 21 Dec 2013 19:16:00 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6D1E01C7D; Sat, 21 Dec 2013 19:16:00 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id rBLJFqA3084907 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 21 Dec 2013 11:15:53 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id rBLJFq3x084906; Sat, 21 Dec 2013 11:15:52 -0800 (PST) (envelope-from jmg) Date: Sat, 21 Dec 2013 11:15:52 -0800 From: John-Mark Gurney To: Guy Yur Subject: Re: 10.0-RC1: net/mpd5 crashes in NgMkSockNode due to stack alignment on ARM EABI Message-ID: <20131221191552.GE99167@funkthat.com> Mail-Followup-To: Guy Yur , freebsd-arm@freebsd.org, freebsd-net@FreeBSD.org References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="/Uq4LBwYP4y1W6pO" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Sat, 21 Dec 2013 11:15:53 -0800 (PST) Cc: freebsd-net@freebsd.org, freebsd-arm@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Dec 2013 19:16:00 -0000 --/Uq4LBwYP4y1W6pO Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Guy Yur wrote this message on Sat, Dec 21, 2013 at 19:24 +0200: > I am running 10.0-RC1 on the BeagleBone Black and the net/mpd5 port is > crashing in libnetgraph NgMkSockNode due to stack alignment. > > 10.0-RC1 World and kernel were compiled in a VirtualBox VM running > 9.2-RELEASE-p2 i386. > clang and ARM_EABI used as the default make options. > > Added prints in NgMkSockNode show rbuf is aligned on 2-byte and not > 4-byte which is needed to access ni->id (a uint32_t). > > ni = 0xbfffe87a > rbuf = 0xbfffe842 > sizeof(resp->header) = 56 > > > (gdb) bt > #0 0x201529a0 in NgMkSockNode (name=, csp=0xbfffe95c, > dsp=0xbfffe958) at /usr/src/lib/libnetgraph/sock.c:134 > #1 0x00037b9c in MppcTestCap () at ccp_mppc.c:754 > #2 0x0007c1f4 in main (ac=4, av=0xbfffeb90) at main.c:248 > #3 0x0000d1b0 in __start (argc=4, argv=0xbfffeb90, env=0xbfffeba4, > ps_strings=, obj=, > cleanup=) at /usr/src/lib/csu/arm/crt1.c:115 > #4 0x203e9dc0 in _thr_ast (curthread=0x200fd000) > at /usr/src/lib/libthr/thread/thr_sig.c:265 > > > Putting rbuf in a union with struct ng_mesg sorted the alignment to > 4-byte and mpd5 didn't crash. > I attached the changes I used to test mpd5 doesn't crash with correct alignment. The patch looks correct, but lets make sure that the -net people don't have an issue with it... I've reattached Guy's patch for review. Guy, bug me in a week or so if I haven't committed it, and I will... Thanks for tracking this down. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." --/Uq4LBwYP4y1W6pO Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="sock-NgMkSockNode.patch" Index: lib/libnetgraph/sock.c =================================================================== --- lib/libnetgraph/sock.c (revision 259250) +++ lib/libnetgraph/sock.c (working copy) @@ -111,8 +111,11 @@ /* Save node name */ strlcpy(namebuf, name, sizeof(namebuf)); } else if (dsp != NULL) { - u_char rbuf[sizeof(struct ng_mesg) + sizeof(struct nodeinfo)]; - struct ng_mesg *const resp = (struct ng_mesg *) rbuf; + union { + u_char rbuf[sizeof(struct ng_mesg) + sizeof(struct nodeinfo)]; + struct ng_mesg res; + } res; + struct ng_mesg *const resp = (struct ng_mesg *) res.rbuf; struct nodeinfo *const ni = (struct nodeinfo *) resp->data; /* Find out the node ID */ @@ -123,7 +126,7 @@ NGLOG("send nodeinfo"); goto errout; } - if (NgRecvMsg(cs, resp, sizeof(rbuf), NULL) < 0) { + if (NgRecvMsg(cs, resp, sizeof(res.rbuf), NULL) < 0) { errnosv = errno; if (_gNgDebugLevel >= 1) NGLOG("recv nodeinfo"); --/Uq4LBwYP4y1W6pO-- From owner-freebsd-net@FreeBSD.ORG Sat Dec 21 21:28:06 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2FCCD2F3; Sat, 21 Dec 2013 21:28:06 +0000 (UTC) Received: from st11p09mm-asmtp001.mac.com (st11p09mm-asmtp001.mac.com [17.164.24.96]) by mx1.freebsd.org (Postfix) with ESMTP id 01154137E; Sat, 21 Dec 2013 21:28:05 +0000 (UTC) Received: from [10.71.14.16] (dsl-hkibrasgw1-58c380-33.dhcp.inet.fi [88.195.128.33]) by st11p09mm-asmtp001.mac.com (Oracle Communications Messaging Server 7u4-27.08(7.0.4.27.7) 64bit (built Aug 22 2013)) with ESMTPSA id <0MY600CXOBID6780@st11p09mm-asmtp001.mac.com>; Sat, 21 Dec 2013 20:27:58 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.11.87,1.0.14,0.0.0000 definitions=2013-12-21_02:2013-12-20,2013-12-21,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=3 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1308280000 definitions=main-1312210160 From: Kimmo Paasiala Content-type: multipart/signed; boundary="Apple-Mail=_F386F8D3-3B94-4215-8B5E-1B29649DC543"; protocol="application/pgp-signature"; micalg=pgp-sha512 Subject: gifconfig_gifX not working with cloned_interfaces? Message-id: <3688B949-7F3B-47FE-8C6A-193805AE2B27@icloud.com> Date: Sat, 21 Dec 2013 22:27:44 +0200 To: freebsd-net@freebsd.org, freebsd-stable@freebsd.org MIME-version: 1.0 (Mac OS X Mail 7.1 \(1827\)) X-Mailer: Apple Mail (2.1827) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Dec 2013 21:28:06 -0000 --Apple-Mail=_F386F8D3-3B94-4215-8B5E-1B29649DC543 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 FreeBSD 10.0-RC2 r259413 i386. I have this set up in rc.conf: cloned_interfaces=3D"gif0" gifconfig_gif0=3D"88.xxx.xxx.xxx 62.yyy.yyy.yyy" ifconfig_gif0_ipv6=3D"inet6 2001:14b8:aaa:bbb::2 2001:14b8:aaa:bbb::1 = prefixlen 128=94 I=92m not using gif_interfaces=3D=93gif0=94 since it=92s deprecated as = per the warning messages spewed by the rc(8) scripts. However this does not work properly The =91ifconfig gif0 tunnel = 88.xxx.xxx.xxx 62.yyy.yyy.yyy=92 does not get executed. It looks to me = that the tunnel set up is only performed when gif0 is listed in = gif_interfaces. I can work around this by doing this instead of the 'gifconfig_gif0' = line: ifconfig_gif0=3D=93 tunnel 88.xxx.xxx.xxx 62.yyy.yyy.yyy=94 Should I file a PR about this? -Kimmo --Apple-Mail=_F386F8D3-3B94-4215-8B5E-1B29649DC543 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJStfnEAAoJEFvLZC0FWRVpMb0H/i3sNYTlfRdIAemYo8rcbWmD rpUDaYJ+KlhGb1PB6ZpWbiqKcSfYtaQ/0t1qlIQ04GY0nTCvk7yFAeob9JiD5qcM fVjfqK4PyoqtwI3p5z9csk7yBx4GM+i5sVkEylcWqzbu0w4KVzBbI9uIlSwxysZb H3nJimbNUrdUXu+HqCGTY8rq+2F7BXKZ3cfhE7IpqHVyVEWamcWT3Tp2UT3kf/pb JOBpnsx6lywEL1cqJjwf+gwn7Pd6ZZwsbyQvF6QC/tsQtYfKZxQ2BGrbFm+BlZKX qwG5StznMU2w5RBms/CqMb/p2oB3cxlTjSySFa50vpq8zld4O9iH4CNzOtmmHIU= =l4mm -----END PGP SIGNATURE----- --Apple-Mail=_F386F8D3-3B94-4215-8B5E-1B29649DC543-- From owner-freebsd-net@FreeBSD.ORG Sat Dec 21 22:46:05 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E14B7A9; Sat, 21 Dec 2013 22:46:05 +0000 (UTC) Received: from mail-oa0-x22c.google.com (mail-oa0-x22c.google.com [IPv6:2607:f8b0:4003:c02::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9A5DC1822; Sat, 21 Dec 2013 22:46:05 +0000 (UTC) Received: by mail-oa0-f44.google.com with SMTP id m1so4402781oag.3 for ; Sat, 21 Dec 2013 14:46:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=dzQ9RyTDate7VGewCv8xctEMpCXvuXrckqOWDSYrIRg=; b=FX5hLXFzrYzm4+DmdC6XI3vdyQ/JLWOXA0SvFgZuO0uyIOboNcj/YWE4bIafHaHtEd NVDwnOlGyPZ+mhpJdKz2VLWobijm3cvOHHR3FtYM7c5H6uav78Q8vG3SR0eLbe5ixphR Cru69Z3Rdh95bbNYcNVxD7NPQYhMTVuH/z9c9MZ6tT4+FNjPUHWFHNvC9S/9/y2Jk+Uv TdHHCRRWG3+5V9uqEA3X+Dmq5X6sd8NH+tzx+eWffSrTj2bXNPJXapVLxoL1kVOoyVjU Q/Tfo3iUTHuWgwhOCQDpsF7O30CD12bjNvyE7l85Fv7oK6meeTFdd3fTLGOLD0h6UaB6 /5nA== MIME-Version: 1.0 X-Received: by 10.60.35.73 with SMTP id f9mr12211155oej.50.1387665964913; Sat, 21 Dec 2013 14:46:04 -0800 (PST) Received: by 10.76.20.82 with HTTP; Sat, 21 Dec 2013 14:46:04 -0800 (PST) In-Reply-To: <20131221191552.GE99167@funkthat.com> References: <20131221191552.GE99167@funkthat.com> Date: Sun, 22 Dec 2013 00:46:04 +0200 Message-ID: Subject: Re: 10.0-RC1: net/mpd5 crashes in NgMkSockNode due to stack alignment on ARM EABI From: Guy Yur To: freebsd-arm@freebsd.org, freebsd-net@freebsd.org Content-Type: multipart/mixed; boundary=089e013c6fa0e84e3904ee132880 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Dec 2013 22:46:06 -0000 --089e013c6fa0e84e3904ee132880 Content-Type: text/plain; charset=UTF-8 On Sat, Dec 21, 2013 at 9:15 PM, John-Mark Gurney wrote: > Guy Yur wrote this message on Sat, Dec 21, 2013 at 19:24 +0200: >> I am running 10.0-RC1 on the BeagleBone Black and the net/mpd5 port is >> crashing in libnetgraph NgMkSockNode due to stack alignment. >> >> 10.0-RC1 World and kernel were compiled in a VirtualBox VM running >> 9.2-RELEASE-p2 i386. >> clang and ARM_EABI used as the default make options. >> >> Added prints in NgMkSockNode show rbuf is aligned on 2-byte and not >> 4-byte which is needed to access ni->id (a uint32_t). >> >> ni = 0xbfffe87a >> rbuf = 0xbfffe842 >> sizeof(resp->header) = 56 >> >> >> (gdb) bt >> #0 0x201529a0 in NgMkSockNode (name=, csp=0xbfffe95c, >> dsp=0xbfffe958) at /usr/src/lib/libnetgraph/sock.c:134 >> #1 0x00037b9c in MppcTestCap () at ccp_mppc.c:754 >> #2 0x0007c1f4 in main (ac=4, av=0xbfffeb90) at main.c:248 >> #3 0x0000d1b0 in __start (argc=4, argv=0xbfffeb90, env=0xbfffeba4, >> ps_strings=, obj=, >> cleanup=) at /usr/src/lib/csu/arm/crt1.c:115 >> #4 0x203e9dc0 in _thr_ast (curthread=0x200fd000) >> at /usr/src/lib/libthr/thread/thr_sig.c:265 >> >> >> Putting rbuf in a union with struct ng_mesg sorted the alignment to >> 4-byte and mpd5 didn't crash. >> I attached the changes I used to test mpd5 doesn't crash with correct alignment. > > The patch looks correct, but lets make sure that the -net people don't > have an issue with it... > > I've reattached Guy's patch for review. > > Guy, bug me in a week or so if I haven't committed it, and I will... > > Thanks for tracking this down. > > -- > John-Mark Gurney Voice: +1 415 225 5579 > > "All that I will do, has been done, All that I have, has not." Hi John, Should I still file a PR? 1. I noticed my patch causes a style bug with the rbuf line now taking 87 columns. 2. Since the union now has a ng_mesg struct, the casting to resp can be skipped and the union member used directly. Attached new patch breaking the rbuf line and swapping resp usage with res.res and &res.res Maybe a different name than res should be used for the union or the member. I only tested the new patch compiles for arm.armv6, haven't verified it. Thanks, Guy --089e013c6fa0e84e3904ee132880 Content-Type: application/octet-stream; name="sock-NgMkSockNode-2.patch" Content-Disposition: attachment; filename="sock-NgMkSockNode-2.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hphga6do1 SW5kZXg6IGxpYi9saWJuZXRncmFwaC9zb2NrLmMKPT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gbGliL2xpYm5ldGdy YXBoL3NvY2suYwkocmV2aXNpb24gMjU5MjUwKQorKysgbGliL2xpYm5ldGdyYXBoL3NvY2suYwko d29ya2luZyBjb3B5KQpAQCAtMTExLDkgKzExMSwxMiBAQAogCQkvKiBTYXZlIG5vZGUgbmFtZSAq LwogCQlzdHJsY3B5KG5hbWVidWYsIG5hbWUsIHNpemVvZihuYW1lYnVmKSk7CiAJfSBlbHNlIGlm IChkc3AgIT0gTlVMTCkgewotCQl1X2NoYXIgcmJ1ZltzaXplb2Yoc3RydWN0IG5nX21lc2cpICsg c2l6ZW9mKHN0cnVjdCBub2RlaW5mbyldOwotCQlzdHJ1Y3QgbmdfbWVzZyAqY29uc3QgcmVzcCA9 IChzdHJ1Y3QgbmdfbWVzZyAqKSByYnVmOwotCQlzdHJ1Y3Qgbm9kZWluZm8gKmNvbnN0IG5pID0g KHN0cnVjdCBub2RlaW5mbyAqKSByZXNwLT5kYXRhOworCQl1bmlvbiB7CisJCQl1X2NoYXIgcmJ1 ZltzaXplb2Yoc3RydWN0IG5nX21lc2cpICsKKwkJCSAgICBzaXplb2Yoc3RydWN0IG5vZGVpbmZv KV07CisJCQlzdHJ1Y3QgbmdfbWVzZyByZXM7CisJCX0gcmVzOworCQlzdHJ1Y3Qgbm9kZWluZm8g KmNvbnN0IG5pID0gKHN0cnVjdCBub2RlaW5mbyAqKSByZXMucmVzLmRhdGE7CiAKIAkJLyogRmlu ZCBvdXQgdGhlIG5vZGUgSUQgKi8KIAkJaWYgKE5nU2VuZE1zZyhjcywgIi4iLCBOR01fR0VORVJJ Q19DT09LSUUsCkBAIC0xMjMsNyArMTI2LDcgQEAKIAkJCQlOR0xPRygic2VuZCBub2RlaW5mbyIp OwogCQkJZ290byBlcnJvdXQ7CiAJCX0KLQkJaWYgKE5nUmVjdk1zZyhjcywgcmVzcCwgc2l6ZW9m KHJidWYpLCBOVUxMKSA8IDApIHsKKwkJaWYgKE5nUmVjdk1zZyhjcywgJnJlcy5yZXMsIHNpemVv ZihyZXMucmJ1ZiksIE5VTEwpIDwgMCkgewogCQkJZXJybm9zdiA9IGVycm5vOwogCQkJaWYgKF9n TmdEZWJ1Z0xldmVsID49IDEpCiAJCQkJTkdMT0coInJlY3Ygbm9kZWluZm8iKTsK --089e013c6fa0e84e3904ee132880--